From 4d3734409d837a0360c1b5fff795deae0cdb2fd7 Mon Sep 17 00:00:00 2001 From: JeffreyChen Date: Fri, 21 Aug 2026 01:07:58 +0800 Subject: [PATCH 01/19] Give three skipped tests the isolation their skip reason asked for MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Their own reason said what they needed: "needs subprocess isolation (see test_actions_menu_gui) ... skip until then". They cover real wiring — a file received on a WebRTC worker thread reaching the GUI thread through a queued signal rather than a thread-affine singleShot, and the admin console's thumbnail poll deleting its QThread each tick instead of leaking one per interval. Skipping was right at the time. Building the WebRTC panel or the admin console and then tearing a worker QThread down aborts the shared pytest process under offscreen Qt, and since deleteLater is a no-op until an event loop runs, the abort lands in some later unrelated file with no traceback. Run all three in one child process that writes a JSON verdict per check and exits without teardown, the shape test_actions_menu_gui already uses. Verdicts are ok / failed / unavailable, so a machine without the webrtc extra reports a skip while one that has it checks the wiring. Confirmed to have teeth rather than assumed: dropping thread.finished.connect(thread.deleteLater) turns the third verdict into "failed: the QThread outlived finish" and leaves the other two green. --- Progress.md | 16 -- README/WHATS_NEW_zh-CN.md | 25 +++ README/WHATS_NEW_zh-TW.md | 25 +++ WHATS_NEW.md | 32 +++ architecture_explore.md | 2 +- .../headless/test_r3_gui_thread_marshal.py | 196 +++++++++++++----- 6 files changed, 231 insertions(+), 65 deletions(-) diff --git a/Progress.md b/Progress.md index fe0d2f81..5f844a12 100644 --- a/Progress.md +++ b/Progress.md @@ -219,22 +219,6 @@ capability enum 值與 variadic `ei_seat_bind_capabilities`、event-type enum --- -## 三個 Qt thread-marshal 測試永久跳過中 - -`TODO` — 需要子行程隔離,做法已知 - -`test/unit_test/headless/test_r3_gui_thread_marshal.py` 裡三個測試被無條件 skip: -`test_panel_signals_expose_file_received`、`test_webrtc_received_file_marshaled_to_gui`、 -`test_thumbnail_poll_thread_is_reaped`。skip 理由自己寫著「needs subprocess -isolation (see test_actions_menu_gui) … skip until then」——那句「until then」就是這條。 - -跟 `CLAUDE.md` §Testing 記的 0xC0000409 `__fastfail` 是同一個家族: -worker→GUI 的 teardown 在共用的 pytest 行程裡把整個監獸帶走。 -`test_actions_menu_gui.py` 已經示範過解法(把建 widget 的部分丟進子行程), -這三個只是還沒改過去。沒改之前,這三條路徑沒有任何回歸保護。 - ---- - ## 兩個講好要爬、還沒爬的門檻 `TODO` — 兩者都寫在 `pyproject.toml` 的註解裡,但不在任何待辦清單上 diff --git a/README/WHATS_NEW_zh-CN.md b/README/WHATS_NEW_zh-CN.md index 5fba5d44..f2ef00e0 100644 --- a/README/WHATS_NEW_zh-CN.md +++ b/README/WHATS_NEW_zh-CN.md @@ -2,6 +2,31 @@ ## 本次更新 (2026-08-20) — 声称支援的平台,這回真的量過了 +### 三个从写出来就一直被跳过的测试 + +`test_r3_gui_thread_marshal.py` 里有三个 `@pytest.mark.skip`,而它们的 skip 理由 +自己就写着需要什么:“needs subprocess isolation (see test_actions_menu_gui) +… skip until then”。它们盖的是真的接线:WebRTC worker 线程收到的文件要通过 +queued signal 回到 GUI 线程(而不是线程绑定的 `QTimer.singleShot`),以及 +admin console 的缩图轮询每一 tick 都要把 `QThread` 删掉,而不是每个间隔漏一个。 + +当时跳过是对的:在**共用的** pytest 进程里建 WebRTC 面板或 admin console、 +再拆掉 worker `QThread`,在 offscreen Qt 下会直接 abort。而因为 `deleteLater` 在没有 +event loop 跑之前是 no-op,那个 abort 甚至不会落在胇事的那支测试上——它会在后面某支 +毫无关系的文件里引爆,而且没有 traceback。 + +**现在它们跑了,在自己的进程里。** 一支 probe 把三项检查一次做完,每项写一个 JSON +结论,然后 `os._exit(0)` 不做 teardown——跟 `test_actions_menu_gui` 对整组标签页用的是 +同一个形状。每项结论是 `ok`、`failed: …` 或 `unavailable: …`,所以没装 `[webrtc]` +extra 的机器(CI 的 `pytest-headless` 就是)得到的是 skip 而不是失败,装了的机器则真的在验接线。 + +这三项检查是有牙的,而且是验过不是假设的:把 `admin_console_tab.py` 里 +`thread.finished.connect(thread.deleteLater)` 那一行拿掉,第三项结论就变成 +`failed: the QThread outlived finish`,而另外两项依旧绿。 + +headless 整套现在不需要任何 `--ignore` 就能从头跑到尾——共 4,815 项通过, +剩下的 skip 全部是可选依赖与平台闸门。里面已经没有任何一句“skip until then”。 + ### Windows arm64 从来不是代码的问题 这一项原本标成 `BLOCKED`,而那只对一半。两个依赖确实没发 diff --git a/README/WHATS_NEW_zh-TW.md b/README/WHATS_NEW_zh-TW.md index 928718be..2f14c60f 100644 --- a/README/WHATS_NEW_zh-TW.md +++ b/README/WHATS_NEW_zh-TW.md @@ -2,6 +2,31 @@ ## 本次更新 (2026-08-20) — 嬣稱支援的平台,這回真的量過了 +### 三個從寫出來就一直被跳過的測試 + +`test_r3_gui_thread_marshal.py` 裡有三個 `@pytest.mark.skip`,而它們的 skip 理由 +自己就寫著需要什麼:「needs subprocess isolation (see test_actions_menu_gui) +… skip until then」。它們蓋的是真的接線:WebRTC worker 執行緒收到的檔案要透過 +queued signal 回到 GUI 執行緒(而不是執行緒綁定的 `QTimer.singleShot`),以及 +admin console 的縮圖輪詢每一 tick 都要把 `QThread` 刪掉,而不是每個間隔漏一個。 + +當時跳過是對的:在**共用的** pytest 行程裡建 WebRTC 面板或 admin console、 +再拆掉 worker `QThread`,在 offscreen Qt 下會直接 abort。而因為 `deleteLater` 在沒有 +event loop 跑之前是 no-op,那個 abort 甚至不會落在胇事的那支測試上——它會在後面某支 +毫無關係的檔案裡引爆,而且沒有 traceback。 + +**現在它們跑了,在自己的行程裡。** 一支 probe 把三項檢查一次做完,每項寫一個 JSON +結論,然後 `os._exit(0)` 不做 teardown——跟 `test_actions_menu_gui` 對整組分頁用的是 +同一個形狀。每項結論是 `ok`、`failed: …` 或 `unavailable: …`,所以沒裝 `[webrtc]` +extra 的機器(CI 的 `pytest-headless` 就是)得到的是 skip 而不是失敗,裝了的機器則真的在驗接線。 + +這三項檢查是有牙的,而且是驗過不是假設的:把 `admin_console_tab.py` 裡 +`thread.finished.connect(thread.deleteLater)` 那一行拿掉,第三項結論就變成 +`failed: the QThread outlived finish`,而另外兩項依舊綠。 + +headless 整套現在不需要任何 `--ignore` 就能從頭跑到尾——共 4,815 項通過, +剩下的 skip 全部是選用相依與平台閘門。裡面已經沒有任何一句「skip until then」。 + ### Windows arm64 從來不是程式的問題 這一項原本標成 `BLOCKED`,而那只對一半。兩個相依確實沒發 diff --git a/WHATS_NEW.md b/WHATS_NEW.md index b20a210c..f61dbddf 100644 --- a/WHATS_NEW.md +++ b/WHATS_NEW.md @@ -2,6 +2,38 @@ ## What's new (2026-08-20) +### Three Tests That Had Been Skipped Since They Were Written + +`test_r3_gui_thread_marshal.py` carried three `@pytest.mark.skip`s whose own +reason said what they needed: *"needs subprocess isolation (see +test_actions_menu_gui) … skip until then."* They covered real wiring — that a +file received on a WebRTC worker thread reaches the GUI thread through a +queued signal rather than a thread-affine `QTimer.singleShot`, and that the +admin console's thumbnail poll deletes its `QThread` each tick instead of +leaking one per interval. + +Skipping them was the right call at the time: building the WebRTC panel or the +admin console and then tearing a worker `QThread` down aborts the *shared* +pytest process under offscreen Qt. Because `deleteLater` is a no-op until an +event loop runs, the abort does not even land in the test that caused it — it +detonates inside some later, unrelated file, with no traceback. + +**They now run, in their own process.** One probe performs all three checks, +writes a JSON verdict per check, and `os._exit(0)`s without teardown — the same +shape `test_actions_menu_gui` has used for the full tab set. Each verdict is +`ok`, `failed: …` or `unavailable: …`, so a machine without the `[webrtc]` +extra (CI's `pytest-headless`, among others) reports a skip rather than a +failure, while a machine that has it actually checks the wiring. + +The checks have teeth, which was verified rather than assumed: deleting the +one line `thread.finished.connect(thread.deleteLater)` from +`admin_console_tab.py` turns the third verdict into `failed: the QThread +outlived finish`, and leaves the other two green. + +The headless suite now runs end to end with no `--ignore` flags — 4,815 +passing, and the only remaining skips are optional-dependency and +platform gates. No "skip until then" is left in it. + ### Windows arm64 Was Never a Code Problem The entry for this said `BLOCKED`, and that was half right. Two dependencies diff --git a/architecture_explore.md b/architecture_explore.md index 5e245314..79bf9d62 100644 --- a/architecture_explore.md +++ b/architecture_explore.md @@ -6,7 +6,7 @@ > 擷取每個模組的 docstring 與頂層公開名稱;統計數字取自實際檔案,非估算。 > 指令數與公開 API 數以 `executor.known_commands()` 與 `je_auto_control.__all__` 在工作樹上實測取得。 > -> **掃描時間**:2026-08-20 **版本**:`pyproject.toml` version `0.0.220` **分支**:`feat/windows-arm64-install` +> **掃描時間**:2026-08-21 **版本**:`pyproject.toml` version `0.0.220` **分支**:`feat/qt-thread-marshal-isolation` --- diff --git a/test/unit_test/headless/test_r3_gui_thread_marshal.py b/test/unit_test/headless/test_r3_gui_thread_marshal.py index a6c073fa..4c735ce9 100644 --- a/test/unit_test/headless/test_r3_gui_thread_marshal.py +++ b/test/unit_test/headless/test_r3_gui_thread_marshal.py @@ -9,8 +9,21 @@ Each test drives the real method from a background ``threading.Thread`` and pumps the GUI event loop, so a queued signal is required for the effect to appear (a thread-affine ``singleShot`` would not fire). + +Three of these run in a subprocess. Constructing the WebRTC panel or the +admin console, then tearing a worker QThread down, aborts the *shared* pytest +process under offscreen Qt (``0xC0000409`` / SIGABRT) — and because +``deleteLater`` is a no-op until an event loop runs, the abort lands inside +some later, unrelated test file. They were skipped outright for that reason; +quarantining the Qt lifetime in a child process is what +``test_actions_menu_gui`` already does, and it is what they needed. The child +writes a JSON verdict per check and ``os._exit(0)``s without teardown. """ +import json import os +import pathlib +import subprocess +import sys import threading import time @@ -19,22 +32,10 @@ os.environ.setdefault("QT_QPA_PLATFORM", "offscreen") pytest.importorskip("PySide6.QtWidgets", exc_type=ImportError) -import shiboken6 # noqa: E402 -from PySide6.QtCore import QEvent, QObject, QThread # noqa: E402 from PySide6.QtWidgets import QApplication # noqa: E402 -# Constructing the WebRTC panel / admin-console QThread teardown inside the -# SHARED pytest process natively aborts (SIGABRT/0xC0000409) under the offscreen -# Qt platform on CI — accumulated Qt state across GUI tests corrupts on teardown. -# The product paths these cover are exercised by the full-widget build in -# test_actions_menu_gui, which is deliberately run in an isolated subprocess for -# exactly this reason. These need the same subprocess isolation before they can -# run in-process; skip until then rather than crash the whole suite. -_OFFSCREEN_SHARED_PROC_ABORT = ( - "worker->GUI teardown aborts the shared pytest process under offscreen Qt; " - "needs subprocess isolation (see test_actions_menu_gui)" -) +REPO_ROOT = pathlib.Path(__file__).resolve().parents[3] @pytest.fixture(scope="module") @@ -93,16 +94,50 @@ def test_presence_registry_event_marshaled_to_gui(qapp): tab.deleteLater() -# --- Finding 8: WebRTC file-received callback ------------------------------ +# --- The three that need their own process -------------------------------- + +# Each check returns "ok", "failed: ...", or "unavailable: ..." so a missing +# optional extra reads as a skip rather than a failure: CI's pytest-headless +# does not install [webrtc], and importing the panel without it raises. +_PROBE = r""" +import json +import os +import pathlib +import sys +import tempfile +import threading +import time + +os.environ.setdefault("QT_QPA_PLATFORM", "offscreen") + +import shiboken6 +from PySide6.QtCore import QEvent, QObject, QThread +from PySide6.QtWidgets import QApplication + +app = QApplication.instance() or QApplication([]) +report = {} + + +def pump_until(predicate, timeout=3.0): + deadline = time.monotonic() + timeout + while not predicate() and time.monotonic() < deadline: + app.processEvents() + time.sleep(0.005) + return predicate() + + +def run_off_thread(target): + thread = threading.Thread(target=target) + thread.start() + thread.join(3.0) -@pytest.mark.skip(reason=_OFFSCREEN_SHARED_PROC_ABORT) -def test_panel_signals_expose_file_received(): + +def check_panel_signals(): from je_auto_control.gui.remote_desktop.webrtc_panel import _PanelSignals - assert hasattr(_PanelSignals(), "file_received") + assert hasattr(_PanelSignals(), "file_received"), "no file_received signal" -@pytest.mark.skip(reason=_OFFSCREEN_SHARED_PROC_ABORT) -def test_webrtc_received_file_marshaled_to_gui(qapp): +def check_webrtc_marshal(): import types from je_auto_control.gui.remote_desktop.webrtc_panel import ( _PanelSignals, _WebRTCViewerPanel, @@ -110,7 +145,7 @@ def test_webrtc_received_file_marshaled_to_gui(qapp): signals = _PanelSignals() - class _Receiver(QObject): + class Receiver(QObject): def __init__(self): super().__init__() self.got = None @@ -118,42 +153,107 @@ def __init__(self): def on_file(self, path): self.got = path - recv = _Receiver() - signals.file_received.connect(recv.on_file) + receiver = Receiver() + signals.file_received.connect(receiver.on_file) stub = types.SimpleNamespace(_signals=signals) - _run_off_thread(lambda: _WebRTCViewerPanel._on_received_file(stub, "file-123")) - assert _pump_until(qapp, lambda: recv.got == "file-123") - + run_off_thread( + lambda: _WebRTCViewerPanel._on_received_file(stub, "file-123")) + assert pump_until(lambda: receiver.got == "file-123"), ( + "the file-received callback never reached the GUI thread; a " + "thread-affine QTimer.singleShot would fail exactly like this") -# --- Finding 9: thumbnail poll thread is reaped on finish ------------------ -@pytest.mark.skip(reason=_OFFSCREEN_SHARED_PROC_ABORT) -def test_thumbnail_poll_thread_is_reaped(qapp, monkeypatch, tmp_path): +def check_thumbnail_reaped(): import je_auto_control.gui.admin_console_tab as admin_mod from je_auto_control.utils.admin.admin_client import AdminConsoleClient - client = AdminConsoleClient(persist_path=tmp_path / "hosts.json") - monkeypatch.setattr(admin_mod, "default_admin_console", lambda: client) + tmp = pathlib.Path(tempfile.mkdtemp()) + client = AdminConsoleClient(persist_path=tmp / "hosts.json") + admin_mod.default_admin_console = lambda: client # Don't run a real background thread: the reaping wiring is what matters, - # and this keeps the test deterministic (no timing, no dangling threads). - monkeypatch.setattr(admin_mod.QThread, "start", lambda self: None) + # and this keeps the check deterministic (no timing, no dangling threads). + admin_mod.QThread.start = lambda self: None tab = admin_mod.AdminConsoleTab() + tab._thumb_timer.stop() + tab._refresh_thumbnails() + + thread = tab._thumb_thread + assert thread is not None, "no thumbnail QThread was created" + assert thread in tab.findChildren(QThread), "thread is not a child of the tab" + + thread.finished.emit() # simulate the QThread finishing + assert tab._thumb_thread is None, "_on_thumb_thread_done did not run" + # Flush the deferred deletions the finished signal scheduled. + app.sendPostedEvents(None, QEvent.Type.DeferredDelete.value) + # Without the deleteLater wiring the QThread would linger as a child of + # the tab, accumulating one per poll tick. + assert not shiboken6.Shiboken.isValid(thread), "the QThread outlived finish" + assert tab.findChildren(QThread) == [], "a QThread lingers as a child" + + +for name, check in [ + ("panel_signals", check_panel_signals), + ("webrtc_marshal", check_webrtc_marshal), + ("thumbnail_reaped", check_thumbnail_reaped), +]: try: - tab._thumb_timer.stop() - tab._refresh_thumbnails() - thread = tab._thumb_thread - assert thread is not None - assert thread in tab.findChildren(QThread) - - thread.finished.emit() # simulate the QThread finishing - assert tab._thumb_thread is None # _on_thumb_thread_done ran - # Flush the deferred deletions the finished signal scheduled. - qapp.sendPostedEvents(None, QEvent.Type.DeferredDelete.value) - # Without the deleteLater wiring the QThread would linger as a child of - # the tab, accumulating one per poll tick. - assert not shiboken6.Shiboken.isValid(thread) - assert tab.findChildren(QThread) == [] - finally: - tab.deleteLater() + check() + except ImportError as error: + report[name] = "unavailable: %s" % (error,) + except AssertionError as error: + report[name] = "failed: %s" % (error or "assertion failed",) + except BaseException as error: + report[name] = "error: %s: %s" % (type(error).__name__, error) + else: + report[name] = "ok" + +sys.stdout.write(json.dumps(report)) +sys.stdout.flush() +# Skip Qt/native-thread teardown entirely -- that teardown is the whole +# reason this runs out of process. The report is already on stdout. +os._exit(0) +""" + + +@pytest.fixture(scope="module") +def marshal_report(): + """Run all three checks in one child process; return its verdicts.""" + env = dict(os.environ, PYTHONPATH=str(REPO_ROOT)) + env.setdefault("QT_QPA_PLATFORM", "offscreen") + # argv is this interpreter plus a module-level literal probe. No shell. + completed = subprocess.run( # nosec B603 # nosemgrep # reason: literal argv, no shell + [sys.executable, "-c", _PROBE], + capture_output=True, text=True, check=False, timeout=180, env=env, + ) + if completed.returncode != 0 or not completed.stdout: + pytest.fail( + "thread-marshal probe subprocess failed " + f"(exit {completed.returncode}):\n{completed.stdout}\n{completed.stderr}" + ) + return json.loads(completed.stdout) + + +def _verdict(report, key): + """Turn one probe verdict into a pass, a skip or a named failure.""" + status = report.get(key) + assert status is not None, f"{key} missing from the probe report: {report}" + if status.startswith("unavailable:"): + pytest.skip(status) + assert status == "ok", status + + +def test_panel_signals_expose_file_received(marshal_report): + """The panel declares the signal the worker hands the file back on.""" + _verdict(marshal_report, "panel_signals") + + +def test_webrtc_received_file_marshaled_to_gui(marshal_report): + """A file received off-thread reaches the GUI thread via a queued signal.""" + _verdict(marshal_report, "webrtc_marshal") + + +def test_thumbnail_poll_thread_is_reaped(marshal_report): + """The thumbnail poll deletes its QThread per tick instead of leaking one.""" + _verdict(marshal_report, "thumbnail_reaped") From c3c972832acbc71e7202e01c01d50393c061b4c4 Mon Sep 17 00:00:00 2001 From: JeffreyChen Date: Fri, 21 Aug 2026 03:24:04 +0800 Subject: [PATCH 02/19] Make the two standing quality thresholds ratchet instead of sit still MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Both were promises living only in a pyproject.toml comment, with nothing that would ever move them. Coverage: fail_under was the first measured baseline and never moved while the suite grew past it, leaving ~15 points unguarded — every square of the matrix clears 50% and CI would still have passed a change deleting a third of the tests. The floor is now the lowest square (50.26%, ubuntu-22.04/3.10), and the comment states the rule that was missing. mypy: the scope was two directories, and a path list only grows when someone remembers to grow it — a new module lands outside it by default. The scope is now the whole package minus a shrink-only list, so 862 of 1,017 files are in the contract and new modules join it on arrival. The verify script fails when a listed module starts passing as loudly as when an unlisted one stops. It also runs win32/linux/darwin rather than only the runner's platform, so the Windows and macOS backends are checked from Ubuntu; 13 modules fail only under a Linux target and 3 only under Windows. The gate has to mean one thing everywhere: a dev checkout with [gui]/[webrtc] disagreed with a bare install about 38 modules, so every non-base third-party module is forced to Any. ignore_missing_imports alone still reads the real package when installed, and follow_imports = "skip" is ignored for .pyi files — PySide6 ships inline stubs, so follow_imports_for_stubs is required too, the same trap the numpy override had already paid for. --- .github/workflows/quality.yml | 14 +- Progress.md | 45 ++++-- WHATS_NEW.md | 57 ++++++++ architecture_explore.md | 9 +- pyproject.toml | 100 +++++++++++++- test/verify/typing_contract_exempt.txt | 173 +++++++++++++++++++++++ test/verify/typing_contract_verify.py | 184 +++++++++++++++++++++++++ 7 files changed, 562 insertions(+), 20 deletions(-) create mode 100644 test/verify/typing_contract_exempt.txt create mode 100644 test/verify/typing_contract_verify.py diff --git a/.github/workflows/quality.yml b/.github/workflows/quality.yml index a7ed579c..ea1a3407 100644 --- a/.github/workflows/quality.yml +++ b/.github/workflows/quality.yml @@ -143,7 +143,7 @@ jobs: ${{ runner.os == 'Linux' && 'xvfb-run -a -s "-screen 0 1280x800x24"' || '' }} pytest -v --tb=short --timeout=120 --cov=je_auto_control --cov-report=term-missing - --cov-report=xml --cov-fail-under=35 + --cov-report=xml --cov-fail-under=50 - name: Upload coverage report uses: actions/upload-artifact@v4 @@ -151,6 +151,12 @@ jobs: name: coverage-${{ matrix.os }}-${{ matrix.python-version }} path: coverage.xml + # The job id is still `typing-stable-api` because it is a required check and + # renaming it silently drops the requirement, but the scope is no longer the + # stable API alone: mypy now checks the whole package, minus the shrink-only + # list in `test/verify/typing_contract_exempt.txt`. The verify script also + # runs the three target platforms mypy can be pointed at, so the Windows and + # macOS backends are checked from this Ubuntu runner rather than skipped. typing-stable-api: runs-on: ubuntu-latest steps: @@ -160,4 +166,8 @@ jobs: python-version: "3.12" - run: pip install -e . # NOSONAR githubactions:S8541,githubactions:S8544 # reason: installs the checked-out project itself, there is no upstream version to lock and the build must run - run: "pip install --only-binary :all: mypy==2.3.0" - - run: mypy je_auto_control/api je_auto_control/utils/failure_bundle + # Deliberately NOT installing the optional extras: the contract forces + # every non-base third-party module to `Any` so the result cannot depend + # on what is installed, and installing them here would only hide a + # regression in that arrangement. + - run: python test/verify/typing_contract_verify.py diff --git a/Progress.md b/Progress.md index 5f844a12..248b0967 100644 --- a/Progress.md +++ b/Progress.md @@ -219,15 +219,42 @@ capability enum 值與 variadic `ei_seat_bind_capabilities`、event-type enum --- -## 兩個講好要爬、還沒爬的門檻 +## 兩個門檻:機制已經有了,剩下的是把數字往前推 -`TODO` — 兩者都寫在 `pyproject.toml` 的註解裡,但不在任何待辦清單上 +`TODO` — 兩者都已經從「講好要爬」變成「有棘輪在擋」,但都還沒到目的地 -- **覆蓋率**:`fail_under = 35`,註解寫著「Raise toward 70 as legacy modules are - brought under the stable API contract」。目標是 70,今天是 35,中間沒有計畫。 -- **mypy 範圍**:CI 只型別檢查兩條路徑(`quality.yml` 的 - `mypy je_auto_control/api je_auto_control/utils/failure_bundle`)。註解寫著 - 「followed legacy modules are analysed for signatures but not reported until they - join the contract」——同樣是講好要擴、還沒擴。 +原本這一條記的是兩個只存在於 `pyproject.toml` 註解裡、沒有任何機制的承諾。 +2026-08-21 把**機制**補上了(做法見 [WHATS_NEW.md](WHATS_NEW.md)),所以這裡改記 +還差多少、以及下一步怎麼走。 -兩者都不是一次做得完的事,但放在這裡至少讓「下一步是什麼」有一個地方可寫。 +### 覆蓋率:地板 50,目標 70 + +`fail_under` 從 35 提到 **50**。35 是第一次實測的基線,之後測試長大了它卻沒動, +於是有 15 個百分點是白讓的:九宮格矩陣每一格都在 50% 以上,而 CI 會放行一個 +把三分之一測試刪掉的改動。現在的地板取自矩陣**最低**的那一格 +(ubuntu-22.04/3.10,50.26%;最高的是 windows-2022/3.14,51.69%)。 + +規則寫進註解了:**這是棘輪,不是目標**——測試賺到了就把地板提上去。 +往 70 的路上還差 20 點,而覆蓋率排除了 `gui/` 與 `language_wrapper/`, +所以剩下的缺口都在 `utils/` 的無頭模組裡。**下一步**是找出跌破平均的大模組 +(`--cov-report=term-missing` 已經開著,CI 每一格都存了 `coverage.xml` artifact), +而不是齊頭式地補測試。 + +### mypy:整包把關,155 個模組還沒過 + +範圍不再是兩條路徑,而是**整包減去一張只准變少的清單** +(`test/verify/typing_contract_exempt.txt`)。差別在於預設值:路徑清單只有人想到才會長, +新模組預設在圈外;現在新模組**預設就在契約裡**,1,017 個檔案有 862 個已經過關。 + +剩下 155 個模組要清。錯誤碼分布是 `attr-defined` 佔大宗,其次 +`arg-type`/`union-attr`/`assignment`,成群集中在 +`utils/remote_desktop`(13)、`gui`(11)、`wrapper`(5)、 +`utils/usb/passthrough`/`utils/mcp_server`/`utils/accessibility/backends`/`linux_wayland`(各 4)。 +**下一步**是一次清一個群集,清完把行從清單刪掉—— +`python test/verify/typing_contract_verify.py --fix` 會替你改,CI 會在你忘了刪的時候紅掉。 + +有一件事別再踩:**這個閘門的判定不能隨環境浮動**。裝了 `[gui]`/`[webrtc]` 的開發機 +與乾淨的 `pip install -e .` 曾經對 38 個模組看法不同(36 個 Qt 模組只在 PySide6 +*不在*時才過關,2 個只在 babel/pytest 不在時才失敗)。修法是把所有非基礎相依的 +第三方模組壓成 `Any`;其中 `follow_imports = "skip"` 對 `.pyi` 無效、必須同時開 +`follow_imports_for_stubs`,正是 numpy 那條註解早就寫過的坑。 diff --git a/WHATS_NEW.md b/WHATS_NEW.md index f61dbddf..9c7f6352 100644 --- a/WHATS_NEW.md +++ b/WHATS_NEW.md @@ -1,5 +1,62 @@ # What's New — AutoControl +## What's new (2026-08-21) + +### Two Quality Gates That Had Been Standing Still + +`Progress.md` carried an entry called "two thresholds we agreed to climb and +never did". Both were promises that lived only in a `pyproject.toml` comment, +with nothing anywhere that would ever move them. Both now have a mechanism. + +**Coverage: the floor was 15 points below what the tests already earn.** +`fail_under = 35` was the first measured baseline and then never moved while the +suite grew past it. Every square of the nine-way matrix is over 50% — measured +on the run for PR #484, from 50.26% (ubuntu-22.04 / 3.10) to 51.69% +(windows-2022 / 3.14) — so CI would have passed a change that deleted a third of +the tests without a word. The floor is now **50**, taken from the lowest square +rather than from one machine, and the comment states the rule that was missing: +this is a ratchet, raised whenever the suite has earned it, not a target to +admire. 70 is still the destination. + +**mypy: the scope was two directories, and could only ever grow by hand.** +The job checked `je_auto_control/api` and `je_auto_control/utils/failure_bundle` +— 4 files — while a comment promised the rest would join "the contract" later. +A scope written as a path list never grows on its own, and every new module +lands outside it by default. + +So the scope is inverted. mypy now checks **the whole package**, and the modules +that do not pass yet are named in `test/verify/typing_contract_exempt.txt`. +**862 of 1,017 files are inside the contract today**, a new module is inside it +the moment it is written, and the list may only shrink: +`test/verify/typing_contract_verify.py` fails just as loudly when a listed +module starts passing (delete the line) as when an unlisted one stops. Both +directions were checked by breaking them. + +**And it checks three platforms, not one.** mypy resolves `sys.platform` +branches against a single target, so the Ubuntu-only job never looked inside +the Windows, macOS or platform-gated code — three of the four backends. That +blind spot was real and is now measured: 13 modules fail only when the target is +Linux, 3 only when it is Windows. The script runs `win32`, `linux` and `darwin` +and unions the results, so the Windows and macOS backends are type-checked from +the Ubuntu runner. + +**The part that took the work: the gate has to mean one thing.** A first +measurement taken on a dev checkout disagreed with a bare `pip install -e .` +about **38 modules** — 36 Qt modules that pass only because PySide6 is *absent*, +and 2 that fail only because `babel` and `pytest` are. Committing that list +would have reddened CI on arrival and mis-listed the other 36. A gate that flips +on `pip install` is not a gate, so every third-party module outside the base +dependency set is now forced to `Any`. `ignore_missing_imports` alone was not +enough — it still lets mypy read the real package when it *is* installed — and +neither was `follow_imports = "skip"`, which is silently ignored for `.pyi` +files: PySide6 ships inline stubs, so `follow_imports_for_stubs` was required +too. That is the same trap the numpy override in `pyproject.toml` had already +paid for and written down. A dev checkout with `[gui]` and `[webrtc]` and a +clean base install now produce identical results. + +The remaining 155 modules cluster — `utils/remote_desktop` (13), `gui` (11), +`wrapper` (5) — and `Progress.md` records clearing them one cluster at a time. + ## What's new (2026-08-20) ### Three Tests That Had Been Skipped Since They Were Written diff --git a/architecture_explore.md b/architecture_explore.md index 79bf9d62..e9c4a17c 100644 --- a/architecture_explore.md +++ b/architecture_explore.md @@ -156,7 +156,7 @@ socket server 有 8 MiB 讀取上限與 30 秒 handler timeout。 | `je_auto_control/__main__.py` | 70 | 舊版 argparse 進入點:`-e` 執行單檔、`-d` 執行整個目錄、`--execute_str` 執行 JSON 字串、`-c` 建立專案。 | | `je_auto_control/cli.py` | 323 | **主 CLI**(`je_auto_control` console script)。子命令:`run`(含 `--var`/`--dry-run`)、`validate`/`lint`、`list-commands`、`fmt`、`record`、`codegen`、`failure-bundle`、`list-jobs`、`start-server`、`start-rest`、`version`。所有子命令延遲匯入,確保不碰 Qt。 | | `je_auto_control/api/__init__.py` | 22 | 版本化整合進入點。 | -| `je_auto_control/api/core.py` | 19 | **穩定無頭 API 門面**:只暴露 `execute_action`、`execute_action_with_vars`、`generate_code`、`run_diagnostics`、`create_failure_bundle`、`failure_bundle_on_error`、`FailureBundleOptions`。mypy 型別契約只針對這一面。 | +| `je_auto_control/api/core.py` | 19 | **穩定無頭 API 門面**:只暴露 `execute_action`、`execute_action_with_vars`、`generate_code`、`run_diagnostics`、`create_failure_bundle`、`failure_bundle_on_error`、`FailureBundleOptions`。mypy 型別契約以此為起點,現已擴到整包(見「設定基線」)。 | | `je_auto_control/utils/deprecation.py` | 35 | 公開 API 的一致性棄用警告。 | | `je_auto_control/utils/http_headers.py` | 32 | 入站 HTTP 標頭的共用防禦式解析。 | | `je_auto_control/utils/sqlite_support.py` | 56 | 選用標準函式庫 `sqlite3` 的取用點:`require_sqlite3()`/`sqlite3_available()`/`SQLITE_ERRORS`。十個以 SQLite 存放狀態的子系統都經由這裡,所以 FreeBSD 這種把 `sqlite3` 另外包成 `databases/py-sqlite3` 的 Python 仍然 import 得起門面。 | @@ -1000,8 +1000,11 @@ GUI 是**選用 extra**(`pip install je_auto_control[gui]`,PySide6 + qt-mate **設定基線**(`pyproject.toml`): - **pytest**:`testpaths` 限定 `test/unit_test/headless` 與 `test/unit_test/flow_control`;`--strict-markers --strict-config`。 -- **coverage**:`fail_under = 35`(實測基線,CI 只確保不退步),排除 `gui/` 與 `language_wrapper/`。 -- **mypy**:只對穩定 API 面把關;`follow_imports = "silent"`,numpy stub 以 `follow_imports_for_stubs` 略過。 +- **coverage**:`fail_under = 50`(棘輪:實測九宮格矩陣最低的一格是 50.26%,取整數當地板),排除 `gui/` 與 `language_wrapper/`。 +- **mypy**:對**整包**把關,尚未過關的模組列在 `test/verify/typing_contract_exempt.txt`(155 個,只准變少); + `test/verify/typing_contract_verify.py` 會分別以 `win32`/`linux`/`darwin` 三個目標平台各跑一次並取聯集, + 所以 Windows 與 macOS 後端在 Ubuntu runner 上也被檢查。非基礎相依的第三方模組一律以 + `follow_imports = "skip"` + `follow_imports_for_stubs` 壓成 `Any`,閘門才不會因為裝了哪個 extra 而改變判定。 - **bandit**:排除 `test`/`docs`/`language_wrapper`(翻譯字典會誤觸 B105),只跳過 B101。 **程式碼硬約束**(CLAUDE.md):循環複雜度 ≤ 10、認知複雜度 ≤ 15、函式 ≤ 75 行、參數 ≤ 7、 diff --git a/pyproject.toml b/pyproject.toml index 9bd5a94a..01205297 100644 --- a/pyproject.toml +++ b/pyproject.toml @@ -137,22 +137,110 @@ omit = ["*/gui/*", "*/language_wrapper/*"] [tool.coverage.report] show_missing = true skip_covered = true -# Initial measured repository baseline. Raise toward 70 as legacy modules are -# brought under the stable API contract; CI enforces that it cannot regress. -fail_under = 35 +# A ratchet, not a target. The rule is that this number is raised to the +# measured minimum whenever the suite has earned it, so coverage the tests +# already provide cannot be given back unnoticed. +# +# 35 was the first measured baseline and then sat still while the suite grew +# past it, which left ~15 points unguarded: every job in the matrix was over +# 50% and CI would still have passed a change that deleted a third of the +# tests. Measured 2026-08-21 on the run for PR #484, lowest square of the +# nine-way matrix (ubuntu-22.04 / 3.10) at 50.26%, highest (windows-2022 / +# 3.14) at 51.69% — so 50 is the floor the whole matrix clears today. +# +# The number is measured, never guessed. Re-read it from the matrix rather +# than from one machine: the spread across platforms is over a point, and +# `--cov-fail-under` in `quality.yml` must hold on the lowest of them. +# +# 70 remains the destination; Progress.md records what stands between here +# and there. +fail_under = 50 [tool.mypy] python_version = "3.10" warn_redundant_casts = true check_untyped_defs = true no_implicit_optional = true -# CI type-checks only the stable API surface; followed legacy modules are -# analysed for signatures but not reported until they join the contract. +# CI type-checks the whole package. The modules that do not pass yet are named +# in `test/verify/typing_contract_exempt.txt`, which may only shrink — see +# `test/verify/typing_contract_verify.py` for what that gate does and why the +# scope is written as "everything minus a list" rather than as a path list. follow_imports = "silent" exclude = "(^test/|^docs/|^build/)" [[tool.mypy.overrides]] -module = ["cv2.*", "Xlib.*", "PySide6.*", "objc.*"] +# Base dependencies that ship no stubs. These are installed by a plain +# `pip install -e .`, so they are present in every environment the gate runs in +# and only their missing type information needs handling. +# Each name is listed twice on purpose: `foo.*` matches submodules only, so the +# bare `foo` is what covers `import foo` (the same trap documented for numpy +# below, which is where it was first paid for). +module = [ + "cv2", "cv2.*", + "je_open_cv", "je_open_cv.*", + "defusedxml", "defusedxml.*", + "mss", "mss.*", +] +ignore_missing_imports = true + +[[tool.mypy.overrides]] +# Every third-party module OUTSIDE the base dependency set, forced to `Any`. +# +# This is what makes the gate mean the same thing everywhere. These modules are +# optional extras (`[gui]`, `[webrtc]`, `[office]`, …) or undeclared per-platform +# backends (pyobjc, Xlib, comtypes, pywin32), so whether mypy can see them +# depends on which extras the machine happens to have installed. Measured: a dev +# checkout with `[gui]` and `[webrtc]` present disagrees with a bare +# `pip install -e .` about 38 modules — 36 Qt modules that pass only because +# PySide6 is absent, and 2 that fail only because babel and pytest are. A gate +# that flips on `pip install` is not a gate, so the contract does not look +# inside any of them. +# +# `follow_imports = "skip"` (not just `ignore_missing_imports`) is the operative +# setting: ignoring a missing import still lets mypy use the real package when +# it IS installed, which is exactly the environment-dependence being removed. +# Our own signatures and logic stay checked; only calls into these libraries go +# unchecked, which is where they already were before the scope widened. +module = [ + # [gui] + "PySide6", "PySide6.*", "qt_material", "qt_material.*", + # [webrtc] / [signaling] / [discovery] + "aiortc", "aiortc.*", "av", "av.*", + "fastapi", "fastapi.*", "uvicorn", "uvicorn.*", + "pydantic", "pydantic.*", "zeroconf", "zeroconf.*", + # [pdf] / [office] / [fuzzy] / [s3] / [locale] / [audio] + "pypdf", "pypdf.*", "openpyxl", "openpyxl.*", + "docx", "docx.*", "pptx", "pptx.*", + "rapidfuzz", "rapidfuzz.*", "boto3", "boto3.*", + "babel", "babel.*", "pycaw", "pycaw.*", + # macOS backends (pyobjc) + "objc", "objc.*", "Quartz", "Quartz.*", "AppKit", "AppKit.*", + "ApplicationServices", "ApplicationServices.*", + "CoreFoundation", "CoreFoundation.*", + # Linux X11 / Windows COM and service backends + "Xlib", "Xlib.*", "comtypes", "comtypes.*", + "win32event", "win32service", "win32serviceutil", + # OCR / vision / LLM / agent backends + "pytesseract", "pytesseract.*", "easyocr", "easyocr.*", + "paddleocr", "paddleocr.*", "anthropic", "anthropic.*", + "openai", "openai.*", + # device, audio and observability backends + "usb", "usb.*", "sounddevice", "sounddevice.*", + "vgamepad", "vgamepad.*", "psutil", "psutil.*", + "opentelemetry", "opentelemetry.*", + "uiautomator2", "uiautomator2.*", "wda", "wda.*", + # dev-time integrations, absent from a runtime install + "pytest", "pytest.*", "behave", "behave.*", + "send2trash", "send2trash.*", + "je_web_runner", "je_web_runner.*", +] +follow_imports = "skip" +# Required, and for the same reason the numpy block below needs it: `skip` is +# ignored for `.pyi` files on its own. PySide6 ships inline stubs, so without +# this the Qt types are read anyway and 26 GUI modules go on failing wherever +# the `[gui]` extra is installed — which is the whole disagreement this block +# exists to remove. +follow_imports_for_stubs = true ignore_missing_imports = true [[tool.mypy.overrides]] diff --git a/test/verify/typing_contract_exempt.txt b/test/verify/typing_contract_exempt.txt new file mode 100644 index 00000000..093a361b --- /dev/null +++ b/test/verify/typing_contract_exempt.txt @@ -0,0 +1,173 @@ +# Modules that do not type-check cleanly yet. +# +# This is the shrink-only exemption list described in +# `typing_contract_verify.py`. mypy checks the whole package; everything named +# here is a module that was already failing when the gate widened to cover it. +# +# Rules: +# * A module may leave this list (fix it, delete the line). That is the point. +# * A module may not join it to make a red build green — fix the types, or +# record why not in Progress.md. +# * The list is measured, never hand-edited: +# python test/verify/typing_contract_verify.py --fix +# +# "Does not type-check" means on at least one of the three targets mypy can be +# pointed at (win32, linux, darwin) — not just the one CI happens to run on. +# +# Measured entries: 155 + +je_auto_control.__main__ +je_auto_control.cli +je_auto_control.gui._auto_click_tab +je_auto_control.gui._image_detect_tab +je_auto_control.gui._record_tab +je_auto_control.gui._report_tab +je_auto_control.gui._screenshot_tab +je_auto_control.gui._script_tab +je_auto_control.gui.assertions_tab +je_auto_control.gui.language_wrapper.multi_language_wrapper +je_auto_control.gui.main_widget +je_auto_control.gui.main_window +je_auto_control.gui.recording_editor_tab +je_auto_control.gui.remote_desktop.webrtc_dialogs +je_auto_control.gui.remote_desktop.webrtc_panel +je_auto_control.gui.test_suite_tab +je_auto_control.linux_wayland._detect +je_auto_control.linux_wayland.capture +je_auto_control.linux_wayland.libei +je_auto_control.linux_wayland.screen +je_auto_control.linux_with_x11.listener.x11_linux_listener +je_auto_control.linux_with_x11.record.x11_linux_record +je_auto_control.linux_with_x11.uinput._device +je_auto_control.osx.keyboard.osx_keyboard +je_auto_control.utils.accessibility.backends.base +je_auto_control.utils.accessibility.backends.linux_backend +je_auto_control.utils.accessibility.backends.windows_backend +je_auto_control.utils.accessibility.backends.windows_query +je_auto_control.utils.accessibility.recorder +je_auto_control.utils.act_modes.act_modes +je_auto_control.utils.actionability.actionability +je_auto_control.utils.admin.admin_client +je_auto_control.utils.agent_memory.agent_memory +je_auto_control.utils.anchor_locator.locator +je_auto_control.utils.annotate.annotate +je_auto_control.utils.app_idle.app_idle +je_auto_control.utils.bulkhead.bulkhead +je_auto_control.utils.change_localize.change_localize +je_auto_control.utils.chatops.router +je_auto_control.utils.clipboard.clipboard +je_auto_control.utils.clipboard.win32_clipboard_api +je_auto_control.utils.clipboard_formats.clipboard_formats +je_auto_control.utils.codegen.codegen +je_auto_control.utils.color_match.color_match +je_auto_control.utils.color_stats.color_stats +je_auto_control.utils.config_sync.client +je_auto_control.utils.critical_exit.critical_exit +je_auto_control.utils.cv2_utils.screen_grabber +je_auto_control.utils.cv2_utils.screen_record +je_auto_control.utils.cv2_utils.video_recording +je_auto_control.utils.dag.runner +je_auto_control.utils.data_quality.data_quality +je_auto_control.utils.dbus_client.session_bus +je_auto_control.utils.deterministic.deterministic +je_auto_control.utils.element_parse.element_parse +je_auto_control.utils.element_proposal.element_proposal +je_auto_control.utils.element_repository.element_repository +je_auto_control.utils.executor.action_executor +je_auto_control.utils.executor.mouse_aliases +je_auto_control.utils.expect_poll.expect_poll +je_auto_control.utils.feature_match.feature_match +je_auto_control.utils.file_assoc.file_assoc +je_auto_control.utils.flake_cluster.flake_cluster +je_auto_control.utils.flow_debugger.flow_debugger +je_auto_control.utils.generate_report.generate_xml_report +je_auto_control.utils.governance.credential_broker +je_auto_control.utils.hotkey.backends.linux_backend +je_auto_control.utils.hotkey.backends.windows_backend +je_auto_control.utils.http_client.http_client +je_auto_control.utils.humanize.motion +je_auto_control.utils.idle_keepawake.idle_keepawake +je_auto_control.utils.json_patch.json_patch +je_auto_control.utils.locale_collation.locale_collation +je_auto_control.utils.lock_session.lock_session +je_auto_control.utils.mcp_server.__main__ +je_auto_control.utils.mcp_server._client_requests +je_auto_control.utils.mcp_server.http_transport +je_auto_control.utils.mcp_server.server +je_auto_control.utils.mcp_server.tools._handlers +je_auto_control.utils.monitor_layout.logical_frame +je_auto_control.utils.mouse_path.mouse_path +je_auto_control.utils.observability.exporter +je_auto_control.utils.observability.metrics +je_auto_control.utils.observability.tracing +je_auto_control.utils.ocr.backends +je_auto_control.utils.plugin_loader.plugin_loader +je_auto_control.utils.preprocess.preprocess +je_auto_control.utils.profiler.resource_profiler +je_auto_control.utils.project.create_project_structure +je_auto_control.utils.pytest_plugin.keywords +je_auto_control.utils.qr.qr +je_auto_control.utils.rate_limit.rate_limit +je_auto_control.utils.recording_edit.editor +je_auto_control.utils.redaction.policies +je_auto_control.utils.redaction.rules +je_auto_control.utils.remote_desktop.audio +je_auto_control.utils.remote_desktop.host_capture +je_auto_control.utils.remote_desktop.host_service +je_auto_control.utils.remote_desktop.jpeg_recorder +je_auto_control.utils.remote_desktop.session_recorder +je_auto_control.utils.remote_desktop.totp +je_auto_control.utils.remote_desktop.video_codec +je_auto_control.utils.remote_desktop.viewer +je_auto_control.utils.remote_desktop.webrtc_host +je_auto_control.utils.remote_desktop.webrtc_host_auth +je_auto_control.utils.remote_desktop.webrtc_host_media +je_auto_control.utils.remote_desktop.webrtc_transport +je_auto_control.utils.remote_desktop.webrtc_viewer +je_auto_control.utils.rest_api.rest_server +je_auto_control.utils.rotated_match.rotated_match +je_auto_control.utils.run_history.history_store +je_auto_control.utils.sbom.sbom +je_auto_control.utils.scheduler.scheduler +je_auto_control.utils.script_vars.interpolate +je_auto_control.utils.search_index.search_index +je_auto_control.utils.secrets.secret_store +je_auto_control.utils.session_guard.session_guard +je_auto_control.utils.socket_server.auto_control_socket_server +je_auto_control.utils.soft_assert.soft_assert +je_auto_control.utils.table_grid_fill.table_grid_fill +je_auto_control.utils.test_record.record_test_class +je_auto_control.utils.text_normalize.text_normalize +je_auto_control.utils.text_regions.text_regions +je_auto_control.utils.tls_acme.challenge +je_auto_control.utils.trash.trash +je_auto_control.utils.triggers.email_trigger +je_auto_control.utils.triggers.trigger_engine +je_auto_control.utils.triggers.webhook_server +je_auto_control.utils.url_canon.url_canon +je_auto_control.utils.usb.passthrough.key_provider +je_auto_control.utils.usb.passthrough.session +je_auto_control.utils.usb.passthrough.viewer_client +je_auto_control.utils.usb.passthrough.winusb_backend +je_auto_control.utils.vision.vlm_api +je_auto_control.utils.visual_regression.compare +je_auto_control.utils.watcher.watcher +je_auto_control.utils.window_capture.window_capture +je_auto_control.utils.window_zorder.window_zorder +je_auto_control.utils.work_queue.work_queue +je_auto_control.utils.xml.xml_file.xml_file +je_auto_control.windows.core.utils.win32_ctype_input +je_auto_control.windows.core.utils.win32_keypress_check +je_auto_control.windows.interception._dll +je_auto_control.windows.interception.mouse +je_auto_control.windows.message.window_message +je_auto_control.windows.mouse.win32_ctype_mouse_control +je_auto_control.windows.record.win32_input_hook +je_auto_control.windows.screen.win32_screen +je_auto_control.windows.window.windows_window_manage +je_auto_control.wrapper.auto_control_keyboard +je_auto_control.wrapper.auto_control_mouse +je_auto_control.wrapper.auto_control_record +je_auto_control.wrapper.auto_control_screen +je_auto_control.wrapper.platform_wrapper +je_auto_control.wrapper.window_backends.base diff --git a/test/verify/typing_contract_verify.py b/test/verify/typing_contract_verify.py new file mode 100644 index 00000000..857013f9 --- /dev/null +++ b/test/verify/typing_contract_verify.py @@ -0,0 +1,184 @@ +"""Type-check the whole package on every platform it supports, against a shrinking list. + +``quality.yml`` used to run ``mypy`` over two paths — ``je_auto_control/api`` and +``je_auto_control/utils/failure_bundle`` — and ``pyproject.toml`` carried a note +saying the rest were "analysed for signatures but not reported until they join +the contract". Nothing was ever going to move a module from *analysed* to +*reported*, because a scope written as an explicit path list only grows when a +human remembers to grow it, and a new module lands outside it by default. + +So the scope is inverted here. mypy checks **the whole package**, and the +modules that do not pass yet are named in ``typing_contract_exempt.txt``. A new +module is therefore inside the contract the moment it is written, and the list +is the only thing standing between today's state and a fully typed package. It +may only shrink: this script fails if a listed module has started passing (go +delete the line) just as loudly as it fails if an unlisted one has stopped. + +The other half of the widening is *where* the check runs. mypy resolves +``sys.platform`` branches against one target platform, so a Linux-only run never +looks inside the Windows, macOS or platform-gated code — three of this project's +four backends. Measured on this tree, that blind spot is real: 13 modules fail +only when the target is Linux and 3 only when it is Windows. This runs all three +targets and unions the results, so a listed module means "does not pass yet on +every supported platform" and a green run means the same thing on any developer +machine as it does on the Ubuntu runner. + +Usage:: + + python test/verify/typing_contract_verify.py # check, exit 1 on drift + python test/verify/typing_contract_verify.py --fix # re-measure the list +""" + +from __future__ import annotations + +import argparse +import json +import subprocess # nosec B404 # reason: runs mypy, a fixed dev-time argv with no shell +import sys +from pathlib import Path + +REPO_ROOT = Path(__file__).resolve().parents[2] +PACKAGE = "je_auto_control" +EXEMPT_FILE = Path(__file__).with_name("typing_contract_exempt.txt") + +# mypy resolves `sys.platform` tests against a single target. The supported +# backends live behind those tests, so every target has to be asked separately. +PLATFORMS = ("win32", "linux", "darwin") + +_HEADER = """\ +# Modules that do not type-check cleanly yet. +# +# This is the shrink-only exemption list described in +# `typing_contract_verify.py`. mypy checks the whole package; everything named +# here is a module that was already failing when the gate widened to cover it. +# +# Rules: +# * A module may leave this list (fix it, delete the line). That is the point. +# * A module may not join it to make a red build green — fix the types, or +# record why not in Progress.md. +# * The list is measured, never hand-edited: +# python test/verify/typing_contract_verify.py --fix +# +# "Does not type-check" means on at least one of the three targets mypy can be +# pointed at ({platforms}) — not just the one CI happens to run on. +# +# Measured entries: {count} +""" + + +def _module_name(relative_path: str) -> str: + """Return the dotted module name for a package-relative source path.""" + parts = Path(relative_path).with_suffix("").parts + if parts and parts[-1] == "__init__": + parts = parts[:-1] + return ".".join(parts) + + +def _failing_modules(platform: str) -> set[str]: + """Return the modules mypy reports errors in when targeting `platform`.""" + completed = subprocess.run( # nosec B603 # reason: fixed argv, no shell, no user input + [sys.executable, "-m", "mypy", "--platform", platform, "-O", "json", PACKAGE], + cwd=REPO_ROOT, + capture_output=True, + text=True, + check=False, + ) + modules: set[str] = set() + for line in completed.stdout.splitlines(): + line = line.strip() + if not line.startswith("{"): + # mypy prints its "Found N errors" summary outside the JSON stream. + continue + try: + record = json.loads(line) + except json.JSONDecodeError: + continue + if record.get("severity") != "error": + continue + path = str(record.get("file", "")).replace("\\", "/") + if path.startswith(f"{PACKAGE}/") or path == f"{PACKAGE}.py": + modules.add(_module_name(path)) + if not modules and completed.returncode not in (0, 1): + raise SystemExit( + f"mypy failed to run for --platform {platform} " + f"(exit {completed.returncode}):\n{completed.stderr.strip()}" + ) + return modules + + +def _measure() -> set[str]: + """Return every module failing on at least one supported target platform.""" + failing: set[str] = set() + for platform in PLATFORMS: + found = _failing_modules(platform) + print(f" --platform {platform}: {len(found)} module(s) with errors") + failing |= found + return failing + + +def _read_exempt() -> set[str]: + """Return the modules named in the committed exemption list.""" + if not EXEMPT_FILE.exists(): + return set() + lines = EXEMPT_FILE.read_text(encoding="utf-8").splitlines() + return {line.strip() for line in lines if line.strip() and not line.startswith("#")} + + +def _write_exempt(modules: set[str]) -> None: + """Rewrite the exemption list from a fresh measurement.""" + header = _HEADER.format(platforms=", ".join(PLATFORMS), count=len(modules)) + body = "".join(f"{module}\n" for module in sorted(modules)) + EXEMPT_FILE.write_text(f"{header}\n{body}", encoding="utf-8") + + +def _report(title: str, modules: list[str], advice: str) -> None: + """Print one drift section.""" + print(f"\n{title} ({len(modules)}):") + for module in modules: + print(f" {module}") + print(f" -> {advice}") + + +def main() -> int: + """Compare the measured failures against the committed list.""" + parser = argparse.ArgumentParser(description=__doc__) + parser.add_argument( + "--fix", + action="store_true", + help="rewrite the exemption list from a fresh measurement", + ) + args = parser.parse_args() + + print(f"Type-checking {PACKAGE} for {len(PLATFORMS)} target platforms...") + failing = _measure() + + if args.fix: + _write_exempt(failing) + print(f"\nWrote {len(failing)} module(s) to {EXEMPT_FILE.name}. Review the diff.") + return 0 + + exempt = _read_exempt() + regressed = sorted(failing - exempt) + fixed = sorted(exempt - failing) + + if not regressed and not fixed: + print(f"\nOK: {len(failing)} module(s) failing, all of them listed.") + return 0 + + if regressed: + _report( + "Modules failing that are not on the list", + regressed, + "fix the type errors; the list may not grow to make this pass", + ) + if fixed: + _report( + "Modules on the list that now pass", + fixed, + "delete these lines: python test/verify/typing_contract_verify.py --fix", + ) + return 1 + + +if __name__ == "__main__": + raise SystemExit(main()) From 7356b12b073315cb2ceb05b1ef72715912210df2 Mon Sep 17 00:00:00 2001 From: JeffreyChen Date: Fri, 21 Aug 2026 06:14:07 +0800 Subject: [PATCH 03/19] Silence the Semgrep audit rule on the mypy invocation Codacy honours a nosemgrep marker only on the exact line it reports, and the audit rule reports the call line. Same shape as adb_client.py: argv is a list of literals plus one value from the module-level PLATFORMS tuple, no shell. --- test/verify/typing_contract_verify.py | 6 +++++- 1 file changed, 5 insertions(+), 1 deletion(-) diff --git a/test/verify/typing_contract_verify.py b/test/verify/typing_contract_verify.py index 857013f9..6d157d77 100644 --- a/test/verify/typing_contract_verify.py +++ b/test/verify/typing_contract_verify.py @@ -76,7 +76,11 @@ def _module_name(relative_path: str) -> str: def _failing_modules(platform: str) -> set[str]: """Return the modules mypy reports errors in when targeting `platform`.""" - completed = subprocess.run( # nosec B603 # reason: fixed argv, no shell, no user input + # The marker has to sit on the `subprocess.run(` line itself: Codacy honours + # `nosemgrep` only on the exact line it reports, and the audit rule reports + # the call, not the argument. See `je_auto_control/android/adb_client.py` + # for the same shape. + completed = subprocess.run( # nosec B603 # nosemgrep: python.lang.security.audit.dangerous-subprocess-use-audit.dangerous-subprocess-use-audit # reason: argv is `sys.executable` plus literals and one value from the module-level PLATFORMS tuple; no shell, no environment, no caller input [sys.executable, "-m", "mypy", "--platform", platform, "-O", "json", PACKAGE], cwd=REPO_ROOT, capture_output=True, From ac9fe7f30382a41a88fecfbb95eeec4f86ce3828 Mon Sep 17 00:00:00 2001 From: JeffreyChen Date: Sat, 22 Aug 2026 03:54:18 +0800 Subject: [PATCH 04/19] Give the platform seam a type contract instead of an accidental one MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit platform_wrapper re-exports eight names and everything above it is written against those names, but nothing said what they were. mypy bound each one to whichever branch it read first — always the Windows one, on every target — so the layer above the seam was checked against Win32 signatures even on Linux and macOS, and a backend could omit a member entirely without a word. The names are now declared before the branches bind them, three of them with Protocols in wrapper/backend_contract.py, and each _platform_* module annotates what it assigns, so an incomplete backend fails in its own file naming the missing member. keyboard and mouse stay Any: their call shape really is per-platform, and Progress.md records what a per-platform protocol would cost. Turning it on surfaced four None-reaching-a-non-optional bugs and one backend disagreeing with the seam it implements; both are in CHANGELOG.md. Six wrapper modules leave the typing exemption list, and four more outside the cluster went green on their own — they had been failing on the seam's accidental types. --- CHANGELOG.md | 31 +++ Progress.md | 43 +++- WHATS_NEW.md | 55 ++++- architecture_explore.md | 36 ++-- .../windows/screen/win32_screen.py | 14 +- je_auto_control/wrapper/_platform_linux.py | 9 +- je_auto_control/wrapper/_platform_osx.py | 9 +- je_auto_control/wrapper/_platform_wayland.py | 9 +- je_auto_control/wrapper/_platform_windows.py | 9 +- .../wrapper/auto_control_keyboard.py | 45 +++- je_auto_control/wrapper/auto_control_mouse.py | 127 ++++++++--- .../wrapper/auto_control_record.py | 7 + .../wrapper/auto_control_screen.py | 16 +- je_auto_control/wrapper/backend_contract.py | 78 +++++++ je_auto_control/wrapper/platform_wrapper.py | 22 ++ .../wrapper/window_backends/base.py | 12 +- .../headless/test_wrapper_seam_contract.py | 197 ++++++++++++++++++ test/verify/typing_contract_exempt.txt | 12 +- 18 files changed, 633 insertions(+), 98 deletions(-) create mode 100644 je_auto_control/wrapper/backend_contract.py create mode 100644 test/unit_test/headless/test_wrapper_seam_contract.py diff --git a/CHANGELOG.md b/CHANGELOG.md index 06d2167a..19311d0f 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -177,6 +177,21 @@ only when documented here with a migration path. ### Changed +- `je_auto_control.stop_record()` returns an empty list where it used to + return `None`. It has always been annotated `-> list`, but the failure path + fell off the end of the function, so a caller that did not write + `stop_record() or []` iterated over `None` and raised in its own code + instead. `stop_record_timeline()` already returned `[]` on the same + failure; the two now agree. +- `je_auto_control.mouse_scroll()` reports its return type as + `Tuple[int, Union[int, str]]`. The value has not changed — X11 and Wayland + still hand back the backend axis code the direction name resolved to, and + every other platform the name itself — the signature just no longer claims + it is always a `str`. +- The Windows screen backend's `size()` returns a `tuple`, not a `list`. + The macOS, X11 and Wayland backends all returned tuples already, and the + public `screen_size()` has always been annotated `Tuple[int, int]`; every + caller unpacks the two values, so nothing that used it needs changing. - The MCP HTTP transport answers `GET /mcp` differently. It used to return `405` with `{"error": "GET stream not supported"}` for every request; it now serves the session's SSE stream when the request carries @@ -357,6 +372,22 @@ only when documented here with a migration path. ### Fixed +- **A backend that could not report the cursor aborted the script instead of + raising what the API promises.** `press_mouse` / `release_mouse` / + `click_mouse` with an omitted `x` or `y` unpacked `get_mouse_position()` + without checking it for `None`, so a backend that answers "I don't know" + raised `TypeError` from the unpacking — outside the + `AutoControlMouseException` family every containment boundary catches. It + now raises `AutoControlMouseException`. `mouse_scroll` reached the same + unpacking through `_scroll_to` and now skips the pre-move instead, which is + the graceful degradation its own comment already documented for backends + that cannot report the cursor. +- **`check_key_is_press()` passed `None` to the backend for an unknown key + name.** A name the virtual-key table has no entry for became `None` and was + handed to the platform backend anyway: a `TypeError` on Windows and a silent + `False` on X11 — that is, "no, it is not pressed" for a key that does not + exist. It now logs the lookup failure and returns `None`, which is the + documented "could not answer" value. - The MCP HTTP transport no longer tries to drain a request body it has already read. Any `4xx` decided *after* the body was parsed — the new unknown-session `404` and duplicate-stream `409`, and the pre-existing diff --git a/Progress.md b/Progress.md index 248b0967..7542abc1 100644 --- a/Progress.md +++ b/Progress.md @@ -240,19 +240,52 @@ capability enum 值與 variadic `ei_seat_bind_capabilities`、event-type enum (`--cov-report=term-missing` 已經開著,CI 每一格都存了 `coverage.xml` artifact), 而不是齊頭式地補測試。 -### mypy:整包把關,155 個模組還沒過 +### mypy:整包把關,145 個模組還沒過 範圍不再是兩條路徑,而是**整包減去一張只准變少的清單** (`test/verify/typing_contract_exempt.txt`)。差別在於預設值:路徑清單只有人想到才會長, -新模組預設在圈外;現在新模組**預設就在契約裡**,1,017 個檔案有 862 個已經過關。 +新模組預設在圈外;現在新模組**預設就在契約裡**,1,017 個檔案有 872 個已經過關。 -剩下 155 個模組要清。錯誤碼分布是 `attr-defined` 佔大宗,其次 +`wrapper` 那一群(6 個模組)在 2026-08-21 清掉了,做法見 [WHATS_NEW.md](WHATS_NEW.md): +平台縫的八個匯出名稱先宣告型別、再讓分支去綁定,其中三個用 +`wrapper/backend_contract.py` 的 Protocol,四個 `_platform_*` 組裝模組各自標注自己綁了什麼。 +順帶帶綠了四個原本卡在縫的偶然型別上、自己其實沒問題的模組 +(`utils/cv2_utils/screen_grabber`、`utils/executor/mouse_aliases`、 +`utils/pytest_plugin/keywords`、`utils/vision/vlm_api`)。 + +剩下 145 個模組要清。錯誤碼分布是 `attr-defined` 佔大宗,其次 `arg-type`/`union-attr`/`assignment`,成群集中在 -`utils/remote_desktop`(13)、`gui`(11)、`wrapper`(5)、 -`utils/usb/passthrough`/`utils/mcp_server`/`utils/accessibility/backends`/`linux_wayland`(各 4)。 +`utils/remote_desktop`(13)、`gui`(11)、 +`linux_wayland`/`utils/accessibility/backends`/`utils/mcp_server`/`utils/usb/passthrough`(各 4)、 +`utils/observability`/`utils/triggers`(各 3)。 **下一步**是一次清一個群集,清完把行從清單刪掉—— `python test/verify/typing_contract_verify.py --fix` 會替你改,CI 會在你忘了刪的時候紅掉。 +#### 平台縫還缺的一半:`keyboard` 與 `mouse` 還沒有合約 + +`TODO` — 八個匯出名稱裡剩這兩個,而它們是被呼叫最多的兩個 + +`screen`/`keyboard_check`/`recorder` 有 Protocol,少一個成員就在該後端自己的檔案裡紅掉。 +`keyboard` 與 `mouse` 維持 `Any`(這也正是 mypy 本來就替它們推出來的型別,沒有變弱), +因為**四個後端的呼叫形狀真的不一樣**——以下是實測簽章: + +| 後端 | `press_key` | `press_mouse` | 滑鼠鍵代碼 | +| --- | --- | --- | --- | +| Windows | `(keycode)` | `(press_button: Tuple[int, int, int])` | 三個 Win32 事件旗標的 tuple | +| macOS | `(keycode, is_shift)`(`is_shift` **沒有預設值**) | `(x, y, mouse_button)` | int | +| X11 | `(keycode)` | `(mouse_keycode)` | int | +| Wayland | `(keycode)` | `(mouse_keycode)` | int | + +一個 Protocol 描述不了這四種,要補起來得每個平台一組、用 mypy 認得的 +`sys.platform` 分支去定義(只認 `== "..."` 與 `.startswith("...")`,`in [...]` **不算**, +已實測)。**前置條件已經備好**:`wrapper/` 裡分辨 macOS 的地方現在一律寫成 +`sys.platform == "darwin"`(與 `is_macos()` 等價但剪得掉),其餘分支問的是輸入堆疊 +(`is_windows()`/`is_x11_unix()`),所以呼叫端這一側不必再改。 + +真正的成本在後端那一側:四個平台的 `keyboard`/`mouse` 模組本身都還在豁免清單上, +Protocol 一旦標上去,它們的內部型別錯誤就會一起浮出來。**所以這條排在 +`linux_wayland`(4)與 `windows/`、`osx/` 那幾個模組清完之後**,不是現在。 + 有一件事別再踩:**這個閘門的判定不能隨環境浮動**。裝了 `[gui]`/`[webrtc]` 的開發機 與乾淨的 `pip install -e .` 曾經對 38 個模組看法不同(36 個 Qt 模組只在 PySide6 *不在*時才過關,2 個只在 babel/pytest 不在時才失敗)。修法是把所有非基礎相依的 diff --git a/WHATS_NEW.md b/WHATS_NEW.md index 9c7f6352..1a37d371 100644 --- a/WHATS_NEW.md +++ b/WHATS_NEW.md @@ -54,8 +54,59 @@ too. That is the same trap the numpy override in `pyproject.toml` had already paid for and written down. A dev checkout with `[gui]` and `[webrtc]` and a clean base install now produce identical results. -The remaining 155 modules cluster — `utils/remote_desktop` (13), `gui` (11), -`wrapper` (5) — and `Progress.md` records clearing them one cluster at a time. +The remaining modules cluster — `utils/remote_desktop` (13), `gui` (11) — and +`Progress.md` records clearing them one cluster at a time. The first cluster is +below. + +### The Platform Seam Now Says What a Backend Is + +`wrapper/platform_wrapper.py` is the Strategy hub: it imports exactly one +backend and re-exports eight names, and everything above it is written against +those names rather than against a platform. Nothing said what they *were*. + +Measured, that had two consequences, and neither was theoretical: + +- **mypy bound every name to the Windows backend, on every target.** The + branches are `if is_windows() / elif is_macos() / …`, which a type checker + cannot resolve, so it read all of them and kept the first — so the layer above + the seam was checked against Win32 signatures even when the target was Linux + or macOS, and it reported "`recorder` has type `OSXRecorder`, expected + `Win32Recorder`" for the *correct* code on the way past. +- **A backend could omit a member and nobody would say so** until a call site + three layers up failed on the user's machine. + +The eight names are now declared before the branches bind them, three of them +with protocols in the new `wrapper/backend_contract.py` — `ScreenBackend`, +`KeyboardCheckBackend`, `RecorderBackend` — and each `_platform_*` assembly +module annotates what it assigns. A backend that does not answer the seam's +questions now fails in its own file, naming the missing member. That is not +hypothetical either: turning it on immediately reported that the Windows +`screen.size()` returns a `list` where the other three backends return a +`tuple` and where the public `screen_size()` promises a tuple. Fixed, and every +caller only ever unpacked the two values. + +`keyboard` and `mouse` stay `Any`, which is what mypy had already inferred for +them: macOS takes `is_shift` on `press_key` and orders its mouse calls +`(x, y, button)` where Windows and X11 take the button alone, and a Windows +mouse "keycode" is a tuple of three event flags where the others are an int. +One protocol cannot describe both, and `Progress.md` carries what it would take. + +Clearing the cluster fixed four bugs the types had been hiding, all of the same +shape — a value that could be `None` reaching something that could not take one. +They are listed in `CHANGELOG.md`. Along the way four modules outside the +cluster (`utils/cv2_utils/screen_grabber`, `utils/executor/mouse_aliases`, +`utils/pytest_plugin/keywords`, `utils/vision/vlm_api`) went green on their own: +they had been failing on the seam's accidental types, not on their own code. +**145 modules left, 872 of 1,017 files inside the contract.** + +Re-measuring `architecture_explore.md` for this change turned up one row the +measuring tool had never been able to read: §8's `wrapper/` row carried prose +in its 行數 cell, so it parsed as a one-column row — the tool wrote the *line* +count into the 檔案數 column, and left the directory out of the named subtotal, +which meant 其餘模組 counted it a second time. The row is now two plain numbers +(19 files, 3,293 lines), the note it was carrying has moved to §5.2 as the +`wrapper/window_backends/` row that section had been missing entirely, and +其餘模組 no longer double-counts 3,293 lines. ## What's new (2026-08-20) diff --git a/architecture_explore.md b/architecture_explore.md index e9c4a17c..b180f899 100644 --- a/architecture_explore.md +++ b/architecture_explore.md @@ -6,7 +6,7 @@ > 擷取每個模組的 docstring 與頂層公開名稱;統計數字取自實際檔案,非估算。 > 指令數與公開 API 數以 `executor.known_commands()` 與 `je_auto_control.__all__` 在工作樹上實測取得。 > -> **掃描時間**:2026-08-21 **版本**:`pyproject.toml` version `0.0.220` **分支**:`feat/qt-thread-marshal-isolation` +> **掃描時間**:2026-08-21 **版本**:`pyproject.toml` version `0.0.220` **分支**:`feat/typing-contract-and-coverage-ratchet` --- @@ -19,8 +19,8 @@ iOS(WebDriverAgent)。核心能力是滑鼠/鍵盤控制、影像辨識、 | 指標 | 數值 | | --- | ---: | -| Python 模組總數(含周邊子專案) | 1,030 | -| 程式碼總行數 | 140,157 | +| Python 模組總數(含周邊子專案) | 1,031 | +| 程式碼總行數 | 140,388 | | `je_auto_control/utils/` 子套件數 | 310 | | `AC_*` 動作指令數(`known_commands()` 實測) | 773 | | 套件門面 `__all__` 公開名稱數 | 1,238 | @@ -167,21 +167,23 @@ socket server 有 8 MiB 讀取上限與 30 秒 handler timeout。 | 模組 | 行數 | 職責 | | --- | ---: | --- | -| `wrapper/platform_wrapper.py` | 73 | **Strategy 樞紐**。依 `sys.platform` 匯入唯一後端並匯出 `keyboard`、`keyboard_check`、`keyboard_keys_table`、`mouse`、`mouse_keys_table`、`special_mouse_keys_table`、`screen`、`recorder`;載入失敗直接拋 `AutoControlException`(fail fast)。 | -| `wrapper/_platform_windows.py` | 325 | Windows 後端組裝:Win32 ctypes 模組 + 虛擬鍵表 + 選用 Interception 驅動。 | -| `wrapper/_platform_osx.py` | 156 | macOS 後端組裝(Quartz 事件 + osx 虛擬鍵表)。 | -| `wrapper/_platform_linux.py` | 267 | X11 後端組裝(python-Xlib + 選用 uinput)。 | -| `wrapper/_platform_wayland.py` | 57 | Wayland 後端組裝(libei/ydotool/grim)。 | -| `wrapper/auto_control_mouse.py` | 366 | 滑鼠 API:位置讀寫、按下/放開/點擊、捲動、座標前處理、送訊息給指定視窗。 | -| `wrapper/auto_control_keyboard.py` | 273 | 鍵盤 API:鍵表查詢、按下/放開/敲擊、`write` 字串、`hotkey` 組合鍵、按鍵狀態偵測。 | -| `wrapper/auto_control_screen.py` | 103 | 螢幕 API:`screen_size`、`screenshot`(可指定區域)、`get_pixel`。 | +| `wrapper/platform_wrapper.py` | 95 | **Strategy 樞紐**。依 `sys.platform` 匯入唯一後端並匯出 `keyboard`、`keyboard_check`、`keyboard_keys_table`、`mouse`、`mouse_keys_table`、`special_mouse_keys_table`、`screen`、`recorder`;八個名稱先以 `backend_contract` 的型別宣告再由分支綁定;載入失敗直接拋 `AutoControlException`(fail fast)。 | +| `wrapper/backend_contract.py` | 78 | 平台縫的型別合約:`ScreenBackend`/`KeyboardCheckBackend`/`RecorderBackend` 三個 Protocol 與 `MouseKeycode` 別名。四個 `_platform_*` 組裝模組各自標注自己綁的是什麼,少一個成員就在該後端自己的檔案裡紅掉,而不是在三層之上的呼叫點。 | +| `wrapper/_platform_windows.py` | 328 | Windows 後端組裝:Win32 ctypes 模組 + 虛擬鍵表 + 選用 Interception 驅動。 | +| `wrapper/_platform_osx.py` | 159 | macOS 後端組裝(Quartz 事件 + osx 虛擬鍵表)。 | +| `wrapper/_platform_linux.py` | 270 | X11 後端組裝(python-Xlib + 選用 uinput)。 | +| `wrapper/_platform_wayland.py` | 60 | Wayland 後端組裝(libei/ydotool/grim)。 | +| `wrapper/auto_control_mouse.py` | 427 | 滑鼠 API:位置讀寫、按下/放開/點擊、捲動、座標前處理、送訊息給指定視窗。 | +| `wrapper/auto_control_keyboard.py` | 304 | 鍵盤 API:鍵表查詢、按下/放開/敲擊、`write` 字串、`hotkey` 組合鍵、按鍵狀態偵測。 | +| `wrapper/auto_control_screen.py` | 111 | 螢幕 API:`screen_size`、`screenshot`(可指定區域)、`get_pixel`。 | | `wrapper/auto_control_image.py` | 83 | 影像 API:`locate_all_image`、`locate_image_center`、`locate_and_click`。 | -| `wrapper/auto_control_record.py` | 107 | 錄製 API:`record`/`stop_record`/`record_to_json`(支援 stop event 與逾時)。 | +| `wrapper/auto_control_record.py` | 114 | 錄製 API:`record`/`stop_record`/`record_to_json`(支援 stop event 與逾時)。 | | `wrapper/auto_control_window.py` | 278 | 視窗管理門面:列舉、尋找、聚焦、等待、關閉、顯示狀態、幾何、所屬行程 PID、依行程列舉/最小化視窗、不搶焦點的投遞式輸入(目前僅 Windows 實作)。 | +| `wrapper/window_backends/` | 985 | 視窗管理的平台縫(`base` / `windows_backend` / `x11_backend` / `macos_backend` / `null_backend`)。放在 `wrapper/` 而不是 `utils/`,因為它必須 import `windows/`、`linux_with_x11/`、`osx/`,而 `utils/` 在分層上在那三者之上。 | ### 5.3 平台後端 -#### Windows(`windows/`,23 檔/1,894 行) +#### Windows(`windows/`,23 檔/1,900 行) | 模組 | 行數 | 職責 | | --- | ---: | --- | @@ -1029,8 +1031,8 @@ socket 預設綁 `127.0.0.1`;資源一律用 `with`。 | `utils/usb/` | 17 | 4,250 | | `je_auto_control/`(頂層 3 檔) | 3 | 2,363 | | `utils/accessibility/` | 13 | 2,818 | -| `wrapper/` | 3,068 | 3,013 新增 `window_backends/`:視窗管理的平台縫(`base` / `windows_backend` / `x11_backend` / `macos_backend` / `null_backend`)。放在 `wrapper/` 而不是 `utils/`,因為它必須 import `windows/`、`linux_with_x11/`、`osx/`,而 `utils/` 在分層上在那三者之上。 | -| `windows/` | 23 | 1,894 | +| `wrapper/` | 19 | 3,293 | +| `windows/` | 23 | 1,900 | | `utils/rest_api/` | 8 | 1,739 | | `utils/agent/` | 8 | 1,250 | | `linux_with_x11/` | 19 | 1,215 | @@ -1042,6 +1044,6 @@ socket 預設綁 `127.0.0.1`;資源一律用 `with`。 | `osx/` | 17 | 907 | | `autocontrol-lsp/` | 8 | 744 | | `utils/hotkey/` | 7 | 727 | -| 其餘模組(約 286 個 `utils/` 子套件 + `android/`/`ios/`/周邊小工具) | 691 | 50,522 | -| **總計** | **1,024** | **140,092** | +| 其餘模組(約 286 個 `utils/` 子套件 + `android/`/`ios/`/周邊小工具) | 673 | 47,454 | +| **總計** | **1,025** | **140,323** | diff --git a/je_auto_control/windows/screen/win32_screen.py b/je_auto_control/windows/screen/win32_screen.py index e0aa268a..2e96a65d 100644 --- a/je_auto_control/windows/screen/win32_screen.py +++ b/je_auto_control/windows/screen/win32_screen.py @@ -1,5 +1,5 @@ import sys -from typing import List, Tuple +from typing import Tuple from je_auto_control.utils.exception.exception_tags import windows_import_error_message from je_auto_control.utils.exception.exceptions import AutoControlException @@ -52,14 +52,20 @@ _CLR_INVALID = 0xFFFFFFFF -def size() -> List[int]: +def size() -> Tuple[int, int]: """ 取得螢幕大小 Get screen size - :return: [width, height] + 一個 tuple,與 osx/x11/wayland 三個後端一致:這裡原本回 list,是四個 + 後端裡唯一一個,而 `wrapper.auto_control_screen.screen_size` 對外承諾的 + 是 tuple。每個呼叫端都只是解包成 width/height,所以型別對齊不改行為。 + The other three backends return a tuple and every caller unpacks the two + values, so this was the odd one out against the seam's own contract. + + :return: (width, height) """ - return [_user32.GetSystemMetrics(0), _user32.GetSystemMetrics(1)] + return _user32.GetSystemMetrics(0), _user32.GetSystemMetrics(1) def get_pixel(x: int, y: int, hwnd: int = 0) -> Tuple[int, int, int]: diff --git a/je_auto_control/wrapper/_platform_linux.py b/je_auto_control/wrapper/_platform_linux.py index 6152c369..88916522 100644 --- a/je_auto_control/wrapper/_platform_linux.py +++ b/je_auto_control/wrapper/_platform_linux.py @@ -70,6 +70,9 @@ from je_auto_control.linux_with_x11.record.x11_linux_record import x11_linux_recorder from je_auto_control.linux_with_x11.screen import x11_linux_screen from je_auto_control.utils.exception.exceptions import AutoControlException +from je_auto_control.wrapper.backend_contract import ( + KeyboardCheckBackend, RecorderBackend, ScreenBackend, +) from je_auto_control.utils.logging.logging_instance import autocontrol_logger autocontrol_logger.info("Load Linux x11 Setting") @@ -259,9 +262,9 @@ def _select_input_backend(): keyboard, mouse = _select_input_backend() -keyboard_check = x11_linux_listener -screen = x11_linux_screen -recorder = x11_linux_recorder +keyboard_check: KeyboardCheckBackend = x11_linux_listener +screen: ScreenBackend = x11_linux_screen +recorder: RecorderBackend = x11_linux_recorder if None in [keyboard_keys_table, mouse_keys_table, special_mouse_keys_table, keyboard, mouse, screen, recorder]: raise AutoControlException("Can't init auto control") diff --git a/je_auto_control/wrapper/_platform_osx.py b/je_auto_control/wrapper/_platform_osx.py index 18a97fe0..0de3cfc4 100644 --- a/je_auto_control/wrapper/_platform_osx.py +++ b/je_auto_control/wrapper/_platform_osx.py @@ -40,6 +40,9 @@ from je_auto_control.osx.record.osx_record import osx_recorder from je_auto_control.osx.screen import osx_screen from je_auto_control.utils.exception.exceptions import AutoControlException +from je_auto_control.wrapper.backend_contract import ( + KeyboardCheckBackend, RecorderBackend, ScreenBackend, +) from je_auto_control.utils.logging.logging_instance import autocontrol_logger autocontrol_logger.info("Load MacOS Setting") @@ -147,10 +150,10 @@ special_mouse_keys_table = None keyboard = osx_keyboard -keyboard_check = osx_keyboard_check +keyboard_check: KeyboardCheckBackend = osx_keyboard_check mouse = osx_mouse -screen = osx_screen -recorder = osx_recorder +screen: ScreenBackend = osx_screen +recorder: RecorderBackend = osx_recorder if None in [keyboard_keys_table, mouse_keys_table, keyboard_check, keyboard, mouse, screen, recorder]: raise AutoControlException("Can't init auto control") diff --git a/je_auto_control/wrapper/_platform_wayland.py b/je_auto_control/wrapper/_platform_wayland.py index 46b5482c..f659aed3 100644 --- a/je_auto_control/wrapper/_platform_wayland.py +++ b/je_auto_control/wrapper/_platform_wayland.py @@ -18,6 +18,9 @@ keyboard_keys_table as _wayland_keyboard_table, ) from je_auto_control.utils.exception.exceptions import AutoControlException +from je_auto_control.wrapper.backend_contract import ( + KeyboardCheckBackend, RecorderBackend, ScreenBackend, +) from je_auto_control.utils.logging.logging_instance import autocontrol_logger @@ -40,9 +43,9 @@ keyboard = wayland_keyboard mouse = wayland_mouse -keyboard_check = wayland_listener -screen = wayland_screen -recorder = wayland_record.wayland_recorder +keyboard_check: KeyboardCheckBackend = wayland_listener +screen: ScreenBackend = wayland_screen +recorder: RecorderBackend = wayland_record.wayland_recorder if None in [keyboard_keys_table, mouse_keys_table, special_mouse_keys_table, diff --git a/je_auto_control/wrapper/_platform_windows.py b/je_auto_control/wrapper/_platform_windows.py index d87ac81a..6a76f805 100644 --- a/je_auto_control/wrapper/_platform_windows.py +++ b/je_auto_control/wrapper/_platform_windows.py @@ -2,6 +2,9 @@ from je_auto_control.utils.exception.exceptions import AutoControlException from je_auto_control.utils.logging.logging_instance import autocontrol_logger +from je_auto_control.wrapper.backend_contract import ( + KeyboardCheckBackend, RecorderBackend, ScreenBackend, +) from je_auto_control.windows.core.utils import win32_keypress_check from je_auto_control.windows.core.utils.win32_vk import ( WIN32_ABSOLUTE, WIN32_EventF_EXTENDEDKEY, WIN32_EventF_KEYUP, @@ -317,9 +320,9 @@ def _build_mouse_keys_table(mouse_module) -> dict: # Build the table only after the backend is chosen; otherwise the # Interception backend would inherit SendInput's flag tuples. mouse_keys_table = _build_mouse_keys_table(mouse) -keyboard_check = win32_keypress_check -screen = win32_screen -recorder = win32_recorder +keyboard_check: KeyboardCheckBackend = win32_keypress_check +screen: ScreenBackend = win32_screen +recorder: RecorderBackend = win32_recorder if None in [keyboard_keys_table, mouse_keys_table, keyboard_check, keyboard, mouse, screen, recorder]: raise AutoControlException("Can't init auto control") diff --git a/je_auto_control/wrapper/auto_control_keyboard.py b/je_auto_control/wrapper/auto_control_keyboard.py index 00e1c225..573ae657 100644 --- a/je_auto_control/wrapper/auto_control_keyboard.py +++ b/je_auto_control/wrapper/auto_control_keyboard.py @@ -1,3 +1,13 @@ +"""Keyboard API: key table, press / release / type, ``write``, hotkeys, state. + +The platform branches follow the rule written out at the top of +``auto_control_mouse``: ask ``platform_id`` which input stack this is instead +of listing OS names (a literal list left the BSDs outside every branch, typing +nothing and reporting success), spell the macOS test as +``sys.platform == "darwin"`` because it is the branch whose signature differs +and the only form a type checker can prune, and raise on a platform that +matches neither. +""" import sys import warnings from typing import Optional, Union, Tuple @@ -10,6 +20,7 @@ AutoControlCantFindKeyException, AutoControlKeyboardException ) from je_auto_control.utils.logging.logging_instance import autocontrol_logger +from je_auto_control.utils.platform_id import is_windows, is_x11_unix from je_auto_control.utils.test_record.record_test_class import record_action_to_list from je_auto_control.utils.text_unicode.text_unicode import unicode_code_units from je_auto_control.wrapper.platform_wrapper import keyboard, keyboard_keys_table, keyboard_check @@ -49,10 +60,16 @@ def press_keyboard_key(keycode: Union[int, str], is_shift: bool = False, autocontrol_logger.info(f"press_keyboard_key, keycode={keycode}, is_shift={is_shift}, skip_record={skip_record}") try: keycode = _resolve_keycode(keycode) - if sys.platform in ["win32", "cygwin", "msys", "linux", "linux2"]: - keyboard.press_key(keycode) - elif sys.platform == "darwin": + # 分支寫法與理由見模組 docstring:非 macOS 問輸入堆疊(BSD 曾經 + # 落在所有分支之外),macOS 用字面比較(型別檢查器剪得掉)。 + # Branch spelling explained in the module docstring. + if sys.platform == "darwin": keyboard.press_key(keycode, is_shift=is_shift) + elif is_windows() or is_x11_unix(): + keyboard.press_key(keycode) + else: + raise AutoControlKeyboardException( + f"press_keyboard_key: no backend for {sys.platform!r}") if not skip_record: record_action_to_list("press_key", {"keycode": keycode, "is_shift": is_shift}) @@ -74,10 +91,16 @@ def release_keyboard_key(keycode: Union[int, str], is_shift: bool = False, autocontrol_logger.info(f"release_keyboard_key, keycode={keycode}, is_shift={is_shift}, skip_record={skip_record}") try: keycode = _resolve_keycode(keycode) - if sys.platform in ["win32", "cygwin", "msys", "linux", "linux2"]: - keyboard.release_key(keycode) - elif sys.platform == "darwin": + # 分支寫法與理由見模組 docstring:非 macOS 問輸入堆疊(BSD 曾經 + # 落在所有分支之外),macOS 用字面比較(型別檢查器剪得掉)。 + # Branch spelling explained in the module docstring. + if sys.platform == "darwin": keyboard.release_key(keycode, is_shift=is_shift) + elif is_windows() or is_x11_unix(): + keyboard.release_key(keycode) + else: + raise AutoControlKeyboardException( + f"release_keyboard_key: no backend for {sys.platform!r}") if not skip_record: record_action_to_list("release_key", {"keycode": keycode, "is_shift": is_shift}) @@ -122,6 +145,14 @@ def check_key_is_press(keycode: Union[int, str]) -> Optional[bool]: autocontrol_logger.info(f"check_key_is_press, keycode={keycode}") try: get_key_code = keycode if isinstance(keycode, int) else keyboard_keys_table.get(keycode) + if get_key_code is None: + # 表裡沒有這個鍵名。原本會把 None 送進後端,讓它自己去炸—— + # Windows 後端會 TypeError,X11 後端則是安靜地回 False。 + # A key name the table has no entry for used to be handed to the + # backend as None: a TypeError on Windows, a silent False on X11. + autocontrol_logger.error( + f"check_key_is_press: {table_cant_find_key_error_message}, keycode={keycode}") + return None record_action_to_list("check_key_is_press", {"keycode": keycode}) return keyboard_check.check_key_is_press(keycode=get_key_code) except (OSError, RuntimeError, AttributeError, TypeError, ValueError) as error: @@ -270,4 +301,4 @@ def send_key_event_to_window(window_title: str, keycode: Union[int, str]) -> Non record_action_to_list("send_key_event_to_window", {"window_title": window_title, "keycode": keycode}, repr(error)) autocontrol_logger.error( f"send_key_event_to_window failed, window={window_title}, keycode={keycode}, error={repr(error)}" - ) \ No newline at end of file + ) diff --git a/je_auto_control/wrapper/auto_control_mouse.py b/je_auto_control/wrapper/auto_control_mouse.py index 9f736fd6..c936e1ff 100644 --- a/je_auto_control/wrapper/auto_control_mouse.py +++ b/je_auto_control/wrapper/auto_control_mouse.py @@ -1,7 +1,29 @@ +"""Mouse API: position, press / release / click, scroll, deprecated posting. + +**How the platform branches in this file are written**, because both spellings +here are load-bearing and neither is arbitrary: + +* Everything that is not macOS asks ``platform_id`` *which input stack* this + is — ``is_windows() or is_x11_unix()`` — rather than listing OS names. The + list this replaced, ``["win32", "cygwin", "msys", "linux", "linux2"]``, left + the BSDs outside every branch: a FreeBSD desktop is an ordinary X11 desktop, + so the call fell off the end, raised nothing, did nothing, and still reported + success. ``mouse_scroll`` was fixed first; the press/release pair carried the + same hole until the seam was typed. +* macOS is spelled ``sys.platform == "darwin"`` rather than ``is_macos()``. + The two are the same test by definition, but only the literal is one a type + checker can resolve, and macOS is the branch whose *signature* differs — it + takes ``(x, y, button)`` where the others take the button alone. Pruning that + branch is what a per-platform backend protocol will need; see + ``wrapper/backend_contract.py`` and ``Progress.md``. +* An OS that matches neither raises rather than returning as if it worked. + ``platform_wrapper`` refuses such a platform at import, so this is the + belt-and-braces half of the same statement. +""" import ctypes import sys import warnings -from typing import Tuple, Union +from typing import Optional, Tuple, Union from je_auto_control.utils.exception.exception_tags import ( mouse_click_mouse_error_message, mouse_get_position_error_message, mouse_press_mouse_error_message, @@ -17,6 +39,7 @@ ) from je_auto_control.utils.test_record.record_test_class import record_action_to_list from je_auto_control.wrapper.auto_control_screen import screen_size +from je_auto_control.wrapper.backend_contract import MouseKeycode from je_auto_control.wrapper.platform_wrapper import mouse, mouse_keys_table, special_mouse_keys_table @@ -28,20 +51,22 @@ def get_mouse_table() -> dict: return mouse_keys_table -def mouse_preprocess(mouse_keycode: Union[int, str], x: int, y: int) -> Tuple[int, int, int]: +def mouse_preprocess(mouse_keycode: Union[int, str], x: Optional[int], + y: Optional[int]) -> Tuple[MouseKeycode, int, int]: """ 前置處理:檢查 keycode 並補齊座標 Preprocess mouse keycode and coordinates :param mouse_keycode: 滑鼠按鍵代碼或字串 Mouse keycode or string - :param x: X 座標 - :param y: Y 座標 + :param x: X 座標,None 代表沿用目前游標位置 + :param y: Y 座標,None 代表沿用目前游標位置 :return: (keycode, x, y) """ + keycode: MouseKeycode = mouse_keycode try: if isinstance(mouse_keycode, str): - mouse_keycode = mouse_keys_table.get(mouse_keycode) - if mouse_keycode is None: + keycode = mouse_keys_table.get(mouse_keycode) + if keycode is None: raise AutoControlCantFindKeyException(table_cant_find_key_error_message) except AutoControlCantFindKeyException as error: raise AutoControlCantFindKeyException(table_cant_find_key_error_message) from error @@ -53,7 +78,15 @@ def mouse_preprocess(mouse_keycode: Union[int, str], x: int, y: int) -> Tuple[in # replay the same effect headlessly. if x is None or y is None: try: - now_x, now_y = get_mouse_position() + position = get_mouse_position() + if position is None: + # 後端回報不出游標位置。原本這裡會在解包時拋 TypeError, + # 而 TypeError 不是這支承諾的例外型別。 + # The backend could not report the cursor: this used to raise + # TypeError from the unpacking, which is not the exception + # this function promises. + raise AutoControlMouseException(mouse_get_position_error_message) + now_x, now_y = position if x is None: x = now_x if y is None: @@ -67,11 +100,7 @@ def mouse_preprocess(mouse_keycode: Union[int, str], x: int, y: int) -> Tuple[in # would hit an un-prototyped SetCursorPos / Xlib fake_input and raise # ctypes.ArgumentError / struct.error — which escapes the executor and # aborts the whole run instead of clicking at the rounded point. - if x is not None: - x = int(x) - if y is not None: - y = int(y) - return mouse_keycode, x, y + return keycode, int(x), int(y) def get_mouse_position() -> tuple[int, int] | None: @@ -124,7 +153,8 @@ def set_mouse_position(x: int, y: int) -> tuple[int, int] | None: raise -def press_mouse(mouse_keycode: Union[int, str], x: int = None, y: int = None) -> tuple[int, int, int] | None: +def press_mouse(mouse_keycode: Union[int, str], x: Optional[int] = None, + y: Optional[int] = None) -> tuple[MouseKeycode, int, int] | None: """ 按下滑鼠按鍵 Press mouse button @@ -135,10 +165,16 @@ def press_mouse(mouse_keycode: Union[int, str], x: int = None, y: int = None) -> param = {"keycode": mouse_keycode, "x": x, "y": y} try: mouse_keycode, x, y = mouse_preprocess(mouse_keycode, x, y) - if sys.platform in ["win32", "cygwin", "msys", "linux", "linux2"]: - mouse.press_mouse(mouse_keycode) - elif sys.platform == "darwin": + # 分支寫法與理由見模組 docstring:非 macOS 問輸入堆疊(BSD 曾經 + # 落在所有分支之外),macOS 用字面比較(型別檢查器剪得掉)。 + # Branch spelling explained in the module docstring. + if sys.platform == "darwin": mouse.press_mouse(x, y, mouse_keycode) + elif is_windows() or is_x11_unix(): + mouse.press_mouse(mouse_keycode) + else: + raise AutoControlMouseException( + f"press_mouse: no backend for {sys.platform!r}") record_action_to_list("press_mouse", param) return mouse_keycode, x, y except AutoControlMouseException as error: @@ -150,7 +186,8 @@ def press_mouse(mouse_keycode: Union[int, str], x: int = None, y: int = None) -> raise -def release_mouse(mouse_keycode: Union[int, str], x: int = None, y: int = None) -> tuple[int, int, int] | None: +def release_mouse(mouse_keycode: Union[int, str], x: Optional[int] = None, + y: Optional[int] = None) -> tuple[MouseKeycode, int, int] | None: """ 放開滑鼠按鍵 Release mouse button @@ -161,10 +198,16 @@ def release_mouse(mouse_keycode: Union[int, str], x: int = None, y: int = None) param = {"keycode": mouse_keycode, "x": x, "y": y} try: mouse_keycode, x, y = mouse_preprocess(mouse_keycode, x, y) - if sys.platform in ["win32", "cygwin", "msys", "linux", "linux2"]: - mouse.release_mouse(mouse_keycode) - elif sys.platform == "darwin": + # 分支寫法與理由見模組 docstring:非 macOS 問輸入堆疊(BSD 曾經 + # 落在所有分支之外),macOS 用字面比較(型別檢查器剪得掉)。 + # Branch spelling explained in the module docstring. + if sys.platform == "darwin": mouse.release_mouse(x, y, mouse_keycode) + elif is_windows() or is_x11_unix(): + mouse.release_mouse(mouse_keycode) + else: + raise AutoControlMouseException( + f"release_mouse: no backend for {sys.platform!r}") record_action_to_list("release_mouse", param) return mouse_keycode, x, y except AutoControlMouseException as error: @@ -176,7 +219,8 @@ def release_mouse(mouse_keycode: Union[int, str], x: int = None, y: int = None) raise -def click_mouse(mouse_keycode: Union[int, str], x: int = None, y: int = None) -> Tuple[int, int, int]: +def click_mouse(mouse_keycode: Union[int, str], x: Optional[int] = None, + y: Optional[int] = None) -> Tuple[MouseKeycode, int, int]: """ 在指定座標按下並放開滑鼠按鍵 Click mouse button at given position @@ -207,7 +251,7 @@ def click_mouse(mouse_keycode: Union[int, str], x: int = None, y: int = None) -> raise AutoControlMouseException(mouse_click_mouse_error_message + " " + repr(error)) from error -def _scroll_to(x: int, y: int) -> None: +def _scroll_to(x: Optional[int], y: Optional[int]) -> None: """ 將游標移到滾動位置,缺漏的座標沿用目前位置並做邊界檢查。 Move the cursor to the requested scroll point, filling in whichever @@ -220,9 +264,12 @@ def _scroll_to(x: int, y: int) -> None: report it (e.g. Wayland) must not be forced to raise. """ width, height = screen_size() + # 兩個座標都給定時不會被讀到,見下面的三元運算。 + # Never read when both coordinates were supplied. + now_x, now_y = 0, 0 if x is None or y is None: try: - now_x, now_y = get_mouse_position() + position = get_mouse_position() except (AutoControlMouseException, NotImplementedError, OSError): # 後端無法回報游標(如 Wayland 會拋 NotImplementedError)時,無法 # 補上缺漏的座標軸,直接略過預先移動,讓滾動發生在目前游標處, @@ -232,15 +279,20 @@ def _scroll_to(x: int, y: int) -> None: # scroll at the current cursor instead of escaping the documented # graceful degradation. return - else: - now_x, now_y = (None, None) + if position is None: + # 同上:回報不出來就別動游標,不要在解包時炸掉整支滾動。 + # Same answer for a backend that returns no position at all. + return + now_x, now_y = position target_x = now_x if x is None else max(0, min(x, width - 1)) target_y = now_y if y is None else max(0, min(y, height - 1)) set_mouse_position(target_x, target_y) -def mouse_scroll(scroll_value: int, x: int = None, y: int = None, - scroll_direction: str = "scroll_down") -> Tuple[int, str]: +def mouse_scroll(scroll_value: int, x: Optional[int] = None, + y: Optional[int] = None, + scroll_direction: str = "scroll_down" + ) -> Tuple[int, Union[int, str]]: """ 模擬滑鼠滾輪操作 Simulate mouse scroll @@ -258,7 +310,10 @@ def mouse_scroll(scroll_value: int, x: int = None, y: int = None, The direction a *positive* count scrolls in. Only the X11 and Wayland backends read it — Windows and macOS have a single wheel axis and take the direction from the sign alone. - :return: (scroll_value, scroll_direction) + :return: (scroll_value, scroll_direction),X11/Wayland 回的是換算後的 + 軸代碼,其餘平台回原字串。 + On X11 and Wayland the direction comes back as the backend axis code + the name resolved to; elsewhere it is the name that was passed in. """ autocontrol_logger.info(f"mouse_scroll, value={scroll_value}, x={x}, y={y}, direction={scroll_direction}") param = {"scroll_value": scroll_value, "x": x, "y": y, "direction": scroll_direction} @@ -280,25 +335,31 @@ def mouse_scroll(scroll_value: int, x: int = None, y: int = None, # another list of OS names: the ["linux", "linux2"] one left the BSDs # outside every branch, so scrolling on FreeBSD raised nothing and # did nothing. + direction: Union[int, str] = scroll_direction if is_windows() or is_macos(): mouse.scroll(scroll_value) elif is_x11_unix(): - scroll_direction = special_mouse_keys_table.get(scroll_direction, scroll_direction) - mouse.scroll(scroll_value, scroll_direction) + # Windows 與 macOS 只有一條滾輪軸,那兩個平台的表是 None。 + # Windows and macOS have a single wheel axis and publish no table. + if special_mouse_keys_table is not None: + direction = special_mouse_keys_table.get(scroll_direction, scroll_direction) + mouse.scroll(scroll_value, direction) else: raise AutoControlMouseException( f"mouse_scroll: no backend for {sys.platform!r}") record_action_to_list("mouse_scroll", param) - return scroll_value, scroll_direction + return scroll_value, direction except AutoControlMouseException as error: autocontrol_logger.error(f"mouse_scroll failed: {repr(error)}") raise AutoControlMouseException(mouse_scroll_error_message + " " + repr(error)) from error -def send_mouse_event_to_window(window, mouse_keycode: Union[int, str], - x: int = None, y: int = None) -> None: +def send_mouse_event_to_window(window: Union[int, str], + mouse_keycode: Union[int, str], + x: Optional[int] = None, + y: Optional[int] = None) -> None: """ 將滑鼠事件送到指定視窗(**已棄用**,改用 ``post_click_to_window``) Send mouse event to a specific window. **Deprecated** — use diff --git a/je_auto_control/wrapper/auto_control_record.py b/je_auto_control/wrapper/auto_control_record.py index df07af6a..62b38b2a 100644 --- a/je_auto_control/wrapper/auto_control_record.py +++ b/je_auto_control/wrapper/auto_control_record.py @@ -35,6 +35,12 @@ def record() -> None: def stop_record() -> list: """ stop current record + + Returns an empty list when the recorder could not be stopped, the same + shape :func:`stop_record_timeline` returns on the same failure: this used + to fall off the end and hand back ``None`` while its signature promised a + list, so every caller that did not write ``stop_record() or []`` iterated + over ``None`` and raised there instead of here. """ autocontrol_logger.info("stop_record") try: @@ -53,6 +59,7 @@ def stop_record() -> list: except (OSError, RuntimeError, AttributeError, TypeError, ValueError, AutoControlException, AutoControlJsonActionException) as error: record_action_to_list("stop_record", None, repr(error)) autocontrol_logger.error(f"stop_record, failed: {repr(error)}") + return [] def stop_record_timeline() -> list: diff --git a/je_auto_control/wrapper/auto_control_screen.py b/je_auto_control/wrapper/auto_control_screen.py index 2a5d25f6..82b7517a 100644 --- a/je_auto_control/wrapper/auto_control_screen.py +++ b/je_auto_control/wrapper/auto_control_screen.py @@ -1,5 +1,5 @@ import sys -from typing import Tuple, List +from typing import List, Optional, Tuple from je_auto_control.utils.cv2_utils.optional import require_cv2 from je_auto_control.utils.cv2_utils.screenshot import pil_screenshot @@ -7,6 +7,7 @@ from je_auto_control.utils.exception.exception_tags import screen_screenshot_error_message from je_auto_control.utils.exception.exceptions import AutoControlScreenException from je_auto_control.utils.logging.logging_instance import autocontrol_logger +from je_auto_control.utils.platform_id import is_windows from je_auto_control.utils.test_record.record_test_class import record_action_to_list from je_auto_control.wrapper.platform_wrapper import screen @@ -29,7 +30,8 @@ def screen_size() -> Tuple[int, int]: raise -def screenshot(file_path: str = None, screen_region: list = None) -> List[int]: +def screenshot(file_path: Optional[str] = None, + screen_region: Optional[list] = None) -> List[int]: """ use to capture current screen 擷取當前螢幕畫面 @@ -89,12 +91,18 @@ def get_pixel(x: int, y: int, hwnd=None): # 因此在這裡給出明確訊息。 # Only the windows backend takes an hwnd. Passing one elsewhere would # surface as a bare TypeError from the backend, so say what is wrong. - if sys.platform not in ["win32", "cygwin", "msys"]: + if not is_windows(): raise AutoControlScreenException( f"get_pixel: hwnd is only supported on Windows, " f"not {sys.platform}" ) - return screen.get_pixel(x, y, hwnd) + # 這一支明著點名 Windows 後端,而不是走平台縫:縫的合約是 (x, y), + # 為了單一平台多出來的第三個參數不該把合約撐開。 + # Named directly rather than called through the seam: the seam's + # contract is (x, y), and one platform's extra argument does not + # belong in it. + from je_auto_control.windows.screen import win32_screen + return win32_screen.get_pixel(x, y, hwnd) except (OSError, RuntimeError, AttributeError, TypeError, ValueError) as error: record_action_to_list("AC_get_pixel", None, repr(error)) autocontrol_logger.error( diff --git a/je_auto_control/wrapper/backend_contract.py b/je_auto_control/wrapper/backend_contract.py new file mode 100644 index 00000000..37557541 --- /dev/null +++ b/je_auto_control/wrapper/backend_contract.py @@ -0,0 +1,78 @@ +"""What the platform seam promises about whichever backend it selected. + +``platform_wrapper`` imports exactly one backend and re-exports its names, so +every module above it is written against *those names* rather than against a +platform. Until now nothing said what they were: mypy bound each name to +whichever branch it read first — always the Windows one, on every target — so +the layer above was silently checked against Win32 signatures even when the +target was Linux or macOS, and a new backend could omit a function entirely +without a word from the type checker. + +These protocols are that missing statement. ``platform_wrapper`` declares its +exports with them and each ``_platform_*`` module annotates what it assigns, so +a backend that does not answer the seam's questions fails where the omission is +— in the backend's own assembly module, naming the missing member — instead of +at some call site three layers up. + +**Why ``keyboard`` and ``mouse`` are not here.** Their call shape is genuinely +platform-specific, which is why every caller branches on ``sys.platform`` before +touching them: macOS takes ``is_shift`` on ``press_key`` and orders its mouse +calls ``(x, y, button)`` where Windows and X11 take the button alone, and a +Windows mouse "keycode" is a tuple of three event flags where the others are a +plain int. One protocol cannot describe both, and the pair is left as ``Any`` +(which is what mypy already inferred for them) until the seam grows a +per-platform protocol for each. See ``Progress.md``. +""" +from typing import Any, Protocol, Tuple + +__all__ = ["KeyboardCheckBackend", "MouseKeycode", "RecorderBackend", + "ScreenBackend"] + +#: 一顆滑鼠鍵在**當前平台**的代碼。X11/Wayland/macOS 是 int,Windows 是 +#: 三個 Win32 事件旗標組成的 tuple——這是 `mouse_keys_table` 的值型別,也是 +#: `press_mouse` 之類的函式收下與回傳的東西。 +#: +#: One mouse button as *this* platform spells it: an int on X11, Wayland and +#: macOS, a tuple of three Win32 event flags on Windows. Named rather than +#: written as a bare ``Any`` so a signature says which kind of unknown it is. +MouseKeycode = Any + + +class ScreenBackend(Protocol): + """Screen geometry and pixel colour, in physical screen coordinates.""" + + def size(self) -> Tuple[int, int]: + """``(width, height)`` of the primary screen.""" + + def get_pixel(self, x: int, y: int) -> Tuple[int, int, int]: + """``(R, G, B)`` at one point of the desktop. + + A backend may accept more than this — the Windows one also takes an + ``hwnd`` — but the seam only promises the two coordinates, so a caller + that wants the extra argument names that backend directly. + """ + + +class KeyboardCheckBackend(Protocol): + """Whether a key is held down right now.""" + + # pylint: disable=too-few-public-methods # reason: one question is the + # whole contract — this backend answers "is that key down?" and nothing else + + def check_key_is_press(self, keycode: int) -> bool: + """``True`` while the key is physically down.""" + + +class RecorderBackend(Protocol): + """Capture of real input events until it is asked to stop. + + ``stop_record`` returns ``Any`` because what it hands back differs by + backend — a ``Queue`` from the shared ``InputRecorder``, a plain list from + the Wayland one — and ``auto_control_record`` already normalises both. + """ + + def record(self) -> None: + """Start capturing keyboard and mouse events.""" + + def stop_record(self) -> Any: + """Stop capturing and return what was captured.""" diff --git a/je_auto_control/wrapper/platform_wrapper.py b/je_auto_control/wrapper/platform_wrapper.py index e844d53e..376a0161 100644 --- a/je_auto_control/wrapper/platform_wrapper.py +++ b/je_auto_control/wrapper/platform_wrapper.py @@ -1,7 +1,29 @@ import sys +from typing import Any, Dict, Optional from je_auto_control.utils.exception.exceptions import AutoControlException from je_auto_control.utils.platform_id import is_bsd, is_macos, is_windows +from je_auto_control.wrapper.backend_contract import ( + KeyboardCheckBackend, RecorderBackend, ScreenBackend, +) + +# 先宣告門面名稱的型別,再讓分支去綁定。 +# The exported names are declared before the branches bind them, so every +# branch is measured against one contract instead of against whichever branch +# mypy happened to read first — which was always the Windows one, on every +# target. `keyboard` and `mouse` are `Any` on purpose; see `backend_contract`. +keyboard: Any +keyboard_check: KeyboardCheckBackend +keyboard_keys_table: Dict[str, int] +mouse: Any +#: Values are platform-specific button codes: a plain int on X11, Wayland and +#: macOS, a tuple of three Win32 event flags on Windows. +mouse_keys_table: Dict[str, Any] +#: X11 and Wayland name their scroll axes here; Windows and macOS have a single +#: wheel axis and publish ``None``. +special_mouse_keys_table: Optional[Dict[str, int]] +screen: ScreenBackend +recorder: RecorderBackend if is_windows(): from je_auto_control.wrapper._platform_windows import ( # noqa: F401 # reason: facade re-export diff --git a/je_auto_control/wrapper/window_backends/base.py b/je_auto_control/wrapper/window_backends/base.py index 0695cebd..9c625e48 100644 --- a/je_auto_control/wrapper/window_backends/base.py +++ b/je_auto_control/wrapper/window_backends/base.py @@ -1,5 +1,5 @@ """Abstract window-management backend.""" -from typing import List, Optional, Tuple +from typing import List, NoReturn, Optional, Tuple from je_auto_control.utils.exception.exceptions import ( AutoControlUnsupportedOperationException, @@ -113,8 +113,14 @@ def post_click(self, window_id: int, button: str, x: int, y: int) -> bool: # --- refusal ----------------------------------------------------------- - def _unsupported(self, operation: str): - """Raise a clear error naming what this backend cannot do.""" + def _unsupported(self, operation: str) -> NoReturn: + """Raise a clear error naming what this backend cannot do. + + ``NoReturn`` is what makes the callers above type-check: every one of + them ends in this call and declares a real return type, which reads as + "falls off the end returning None" unless the checker is told this + never comes back. + """ raise AutoControlUnsupportedOperationException( f"{operation} is not supported by the {self.name} window backend", ) diff --git a/test/unit_test/headless/test_wrapper_seam_contract.py b/test/unit_test/headless/test_wrapper_seam_contract.py new file mode 100644 index 00000000..8b8a0c26 --- /dev/null +++ b/test/unit_test/headless/test_wrapper_seam_contract.py @@ -0,0 +1,197 @@ +"""What the platform seam promises, exercised on every platform from one host. + +``wrapper/platform_wrapper.py`` picks one backend and re-exports its names; +everything above it is written against those names. These tests stand a +recording stub in each name's place and drive the wrapper with ``sys.platform`` +set to somebody else's OS, so a branch that reaches no backend — the failure +mode that is invisible on the developer's own machine — fails here. + +They also pin the answers the wrapper must give when a backend cannot answer: +"I don't know where the cursor is" and "there is no such key" have documented +results, and both used to be a bare ``TypeError`` from an unpacking or a +``None`` handed to a native call. +""" +import sys +import types + +import pytest + +from je_auto_control.utils.exception.exceptions import AutoControlMouseException +from je_auto_control.wrapper import ( + auto_control_keyboard, auto_control_mouse, auto_control_record, +) + +# The BSDs run the same X11 stack as Linux, and ``sys.platform`` there carries +# the major version — which is why a literal ["linux", "linux2"] list missed it. +BSD = "freebsd14" + + +@pytest.fixture() +def keyboard_env(monkeypatch): + """A recording keyboard backend, with the key table and recorder silenced.""" + calls: list = [] + backend = types.ModuleType("stub_keyboard") + backend.press_key = lambda keycode, **kwargs: calls.append( + ("press", keycode, kwargs)) + backend.release_key = lambda keycode, **kwargs: calls.append( + ("release", keycode, kwargs)) + monkeypatch.setattr(auto_control_keyboard, "keyboard", backend) + monkeypatch.setattr(auto_control_keyboard, "keyboard_keys_table", {"a": 65}) + monkeypatch.setattr(auto_control_keyboard, "record_action_to_list", + lambda *a, **k: None) + return calls + + +@pytest.mark.parametrize("platform", ["win32", "cygwin", "msys", "linux", + "linux2", BSD]) +def test_a_key_press_reaches_the_backend_on_every_x11_and_win32_platform( + keyboard_env, monkeypatch, platform): + """Regression: a BSD matched no branch, so no key was pressed. + + ``press_keyboard_key`` tested ``sys.platform`` against a literal list of + Linux and Windows names and then against ``darwin``. On FreeBSD it fell + off the end: nothing was pressed, nothing was raised, and the keycode came + back as if it had worked. ``mouse_scroll`` had the same hole and was fixed; + this is the same family. + """ + monkeypatch.setattr(sys, "platform", platform) + + assert auto_control_keyboard.press_keyboard_key("a") == "65" + assert keyboard_env == [("press", 65, {})] + + +@pytest.mark.parametrize("platform", ["win32", "linux", BSD]) +def test_a_key_release_reaches_the_backend_too(keyboard_env, monkeypatch, + platform): + """The release path carried the identical list, and the identical hole.""" + monkeypatch.setattr(sys, "platform", platform) + + auto_control_keyboard.release_keyboard_key("a") + + assert keyboard_env == [("release", 65, {})] + + +def test_macos_still_gets_its_shift_argument(keyboard_env, monkeypatch): + """macOS is the one backend whose press_key takes ``is_shift``.""" + monkeypatch.setattr(sys, "platform", "darwin") + + auto_control_keyboard.press_keyboard_key("a", is_shift=True) + + assert keyboard_env == [("press", 65, {"is_shift": True})] + + +def test_an_unknown_key_name_never_reaches_the_backend(keyboard_env, + monkeypatch): + """Regression: the table miss was handed to the backend as ``None``. + + ``check_key_is_press("no_such_key")`` looked the name up, got ``None`` and + passed it on: a ``TypeError`` on Windows, and on X11 a silent ``False`` — + "that key is not pressed" for a key that does not exist. + """ + checked: list = [] + monkeypatch.setattr( + auto_control_keyboard, "keyboard_check", + types.SimpleNamespace( + check_key_is_press=lambda keycode: checked.append(keycode) or True)) + + assert auto_control_keyboard.check_key_is_press("no_such_key") is None + assert checked == [] + # A key the table does know still gets through untouched. + assert auto_control_keyboard.check_key_is_press("a") is True + assert checked == [65] + + +@pytest.fixture() +def mouse_env(monkeypatch): + """A recording mouse backend with an X11-shaped button table.""" + calls: list = [] + backend = types.ModuleType("stub_mouse") + backend.press_mouse = lambda *args: calls.append(("press",) + args) + backend.release_mouse = lambda *args: calls.append(("release",) + args) + backend.scroll = lambda *args: calls.append(("scroll",) + args) + backend.set_position = lambda x, y: calls.append(("move", x, y)) + monkeypatch.setattr(auto_control_mouse, "mouse", backend) + monkeypatch.setattr(auto_control_mouse, "mouse_keys_table", + {"mouse_left": 1}) + monkeypatch.setattr(auto_control_mouse, "record_action_to_list", + lambda *a, **k: None) + monkeypatch.setattr(auto_control_mouse, "screen_size", lambda: (1920, 1080)) + return calls + + +@pytest.mark.parametrize("platform", ["win32", "cygwin", "linux", BSD]) +def test_a_button_press_reaches_the_backend_on_a_bsd_too(mouse_env, monkeypatch, + platform): + """The same missing-branch hole, in ``press_mouse`` and ``release_mouse``.""" + monkeypatch.setattr(sys, "platform", platform) + + auto_control_mouse.press_mouse("mouse_left", 10, 20) + auto_control_mouse.release_mouse("mouse_left", 10, 20) + + assert mouse_env == [("press", 1), ("release", 1)] + + +def test_macos_keeps_its_xy_first_button_order(mouse_env, monkeypatch): + monkeypatch.setattr(sys, "platform", "darwin") + + auto_control_mouse.press_mouse("mouse_left", 10, 20) + + assert mouse_env == [("press", 10, 20, 1)] + + +def test_an_unreportable_cursor_raises_the_documented_exception(mouse_env, + monkeypatch): + """Regression: ``None`` from the backend was unpacked, raising TypeError. + + A backend that cannot report the cursor answers ``None``. That went + straight into ``now_x, now_y = ...``, so the caller got a ``TypeError`` + from outside the ``AutoControlException`` family every containment + boundary catches — instead of the exception this API documents. + """ + monkeypatch.setattr(sys, "platform", "linux") + monkeypatch.setattr(auto_control_mouse, "get_mouse_position", lambda: None) + + with pytest.raises(AutoControlMouseException): + auto_control_mouse.press_mouse("mouse_left") + + assert mouse_env == [] + + +def test_scrolling_survives_an_unreportable_cursor(mouse_env, monkeypatch): + """One coordinate given and no cursor to fill the other: scroll anyway. + + Skipping the pre-move is the graceful degradation ``_scroll_to`` already + documented for backends that cannot report the cursor; it used to reach + the same unpacking and raise ``TypeError`` out of the whole call. + """ + monkeypatch.setattr(sys, "platform", "linux") + monkeypatch.setattr(auto_control_mouse, "special_mouse_keys_table", + {"scroll_down": 5}) + monkeypatch.setattr(auto_control_mouse, "get_mouse_position", lambda: None) + + auto_control_mouse.mouse_scroll(3, x=100) + + assert mouse_env == [("scroll", 3, 5)] + + +def test_scrolling_reads_no_axis_table_where_there_is_none(mouse_env, + monkeypatch): + """Windows and macOS have one wheel axis and publish no axis table.""" + monkeypatch.setattr(sys, "platform", "win32") + monkeypatch.setattr(auto_control_mouse, "special_mouse_keys_table", None) + + assert auto_control_mouse.mouse_scroll(-2) == (-2, "scroll_down") + assert mouse_env == [("scroll", -2)] + + +def test_stop_record_returns_a_list_when_the_recorder_fails(monkeypatch): + """Regression: it returned ``None`` while its signature promised a list.""" + def _explode(): + raise RuntimeError("no session") + + monkeypatch.setattr(auto_control_record, "recorder", + types.SimpleNamespace(stop_record=_explode)) + monkeypatch.setattr(auto_control_record, "record_action_to_list", + lambda *a, **k: None) + + assert auto_control_record.stop_record() == [] diff --git a/test/verify/typing_contract_exempt.txt b/test/verify/typing_contract_exempt.txt index 093a361b..87abee44 100644 --- a/test/verify/typing_contract_exempt.txt +++ b/test/verify/typing_contract_exempt.txt @@ -14,7 +14,7 @@ # "Does not type-check" means on at least one of the three targets mypy can be # pointed at (win32, linux, darwin) — not just the one CI happens to run on. # -# Measured entries: 155 +# Measured entries: 145 je_auto_control.__main__ je_auto_control.cli @@ -63,7 +63,6 @@ je_auto_control.utils.color_match.color_match je_auto_control.utils.color_stats.color_stats je_auto_control.utils.config_sync.client je_auto_control.utils.critical_exit.critical_exit -je_auto_control.utils.cv2_utils.screen_grabber je_auto_control.utils.cv2_utils.screen_record je_auto_control.utils.cv2_utils.video_recording je_auto_control.utils.dag.runner @@ -74,7 +73,6 @@ je_auto_control.utils.element_parse.element_parse je_auto_control.utils.element_proposal.element_proposal je_auto_control.utils.element_repository.element_repository je_auto_control.utils.executor.action_executor -je_auto_control.utils.executor.mouse_aliases je_auto_control.utils.expect_poll.expect_poll je_auto_control.utils.feature_match.feature_match je_auto_control.utils.file_assoc.file_assoc @@ -105,7 +103,6 @@ je_auto_control.utils.plugin_loader.plugin_loader je_auto_control.utils.preprocess.preprocess je_auto_control.utils.profiler.resource_profiler je_auto_control.utils.project.create_project_structure -je_auto_control.utils.pytest_plugin.keywords je_auto_control.utils.qr.qr je_auto_control.utils.rate_limit.rate_limit je_auto_control.utils.recording_edit.editor @@ -149,7 +146,6 @@ je_auto_control.utils.usb.passthrough.key_provider je_auto_control.utils.usb.passthrough.session je_auto_control.utils.usb.passthrough.viewer_client je_auto_control.utils.usb.passthrough.winusb_backend -je_auto_control.utils.vision.vlm_api je_auto_control.utils.visual_regression.compare je_auto_control.utils.watcher.watcher je_auto_control.utils.window_capture.window_capture @@ -165,9 +161,3 @@ je_auto_control.windows.mouse.win32_ctype_mouse_control je_auto_control.windows.record.win32_input_hook je_auto_control.windows.screen.win32_screen je_auto_control.windows.window.windows_window_manage -je_auto_control.wrapper.auto_control_keyboard -je_auto_control.wrapper.auto_control_mouse -je_auto_control.wrapper.auto_control_record -je_auto_control.wrapper.auto_control_screen -je_auto_control.wrapper.platform_wrapper -je_auto_control.wrapper.window_backends.base From 7867900e3cc374212f5d87d691995b7b72727f99 Mon Sep 17 00:00:00 2001 From: JeffreyChen Date: Sat, 22 Aug 2026 04:07:45 +0800 Subject: [PATCH 05/19] Clear the linux_wayland cluster off the typing contract MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Thirty-three of the forty errors were one sentence: libei reads its resolved entry points straight off an Optional[BoundSymbols]. Every one of those reads is guarded — connect() refuses an unavailable backend, _emit a disconnected one — but the guarantee lived in the call graph rather than anywhere a reader or a checker could see it, and a new call site that skipped a guard would raise AttributeError, which is not an AutoControlException and so escapes every containment boundary. They now go through one _api property that raises LibeiUnavailable, which is what this module says every failure in it raises. _teardown keeps the raw attribute on purpose: it runs from an except BaseException handler, where raising would replace the real failure. The other three were each one honest disagreement — environment probes typed dict while defaulting to os.environ, a writer declared to return None while every caller returns the tool's stdout, and Pillow's getpixel union handed to callers that unpack three ints. No behaviour changes. --- Progress.md | 20 +++-- WHATS_NEW.md | 32 ++++++++ architecture_explore.md | 14 ++-- je_auto_control/linux_wayland/_detect.py | 6 +- je_auto_control/linux_wayland/capture.py | 9 ++- je_auto_control/linux_wayland/libei.py | 93 +++++++++++++++--------- je_auto_control/linux_wayland/screen.py | 10 ++- test/verify/typing_contract_exempt.txt | 6 +- 8 files changed, 130 insertions(+), 60 deletions(-) diff --git a/Progress.md b/Progress.md index 7542abc1..5cb35b7e 100644 --- a/Progress.md +++ b/Progress.md @@ -240,11 +240,11 @@ capability enum 值與 variadic `ei_seat_bind_capabilities`、event-type enum (`--cov-report=term-missing` 已經開著,CI 每一格都存了 `coverage.xml` artifact), 而不是齊頭式地補測試。 -### mypy:整包把關,145 個模組還沒過 +### mypy:整包把關,141 個模組還沒過 範圍不再是兩條路徑,而是**整包減去一張只准變少的清單** (`test/verify/typing_contract_exempt.txt`)。差別在於預設值:路徑清單只有人想到才會長, -新模組預設在圈外;現在新模組**預設就在契約裡**,1,017 個檔案有 872 個已經過關。 +新模組預設在圈外;現在新模組**預設就在契約裡**,1,017 個檔案有 876 個已經過關。 `wrapper` 那一群(6 個模組)在 2026-08-21 清掉了,做法見 [WHATS_NEW.md](WHATS_NEW.md): 平台縫的八個匯出名稱先宣告型別、再讓分支去綁定,其中三個用 @@ -253,10 +253,17 @@ capability enum 值與 variadic `ei_seat_bind_capabilities`、event-type enum (`utils/cv2_utils/screen_grabber`、`utils/executor/mouse_aliases`、 `utils/pytest_plugin/keywords`、`utils/vision/vlm_api`)。 -剩下 145 個模組要清。錯誤碼分布是 `attr-defined` 佔大宗,其次 +`linux_wayland` 那一群(4 個模組)在 2026-08-22 清掉了:`libei` 的 33 處 +`Optional[BoundSymbols]` 改走一個會拋 `LibeiUnavailable` 的存取器(`_api`), +`_detect` 的兩支環境查詢收 `Mapping[str, str]` 而不是 `dict`(預設值就是 `os.environ`), +`capture._write_to_temp_png` 承認它的 writer 會回傳東西而它不看, +`screen.get_pixel` 把 Pillow 那個「每種 mode 的答案聯集」收斂成它自己承諾的三元組。 +做法見 [WHATS_NEW.md](WHATS_NEW.md)。 + +剩下 141 個模組要清。錯誤碼分布是 `attr-defined` 佔大宗,其次 `arg-type`/`union-attr`/`assignment`,成群集中在 `utils/remote_desktop`(13)、`gui`(11)、 -`linux_wayland`/`utils/accessibility/backends`/`utils/mcp_server`/`utils/usb/passthrough`(各 4)、 +`utils/accessibility/backends`/`utils/mcp_server`/`utils/usb/passthrough`(各 4)、 `utils/observability`/`utils/triggers`(各 3)。 **下一步**是一次清一個群集,清完把行從清單刪掉—— `python test/verify/typing_contract_verify.py --fix` 會替你改,CI 會在你忘了刪的時候紅掉。 @@ -283,8 +290,9 @@ capability enum 值與 variadic `ei_seat_bind_capabilities`、event-type enum (`is_windows()`/`is_x11_unix()`),所以呼叫端這一側不必再改。 真正的成本在後端那一側:四個平台的 `keyboard`/`mouse` 模組本身都還在豁免清單上, -Protocol 一旦標上去,它們的內部型別錯誤就會一起浮出來。**所以這條排在 -`linux_wayland`(4)與 `windows/`、`osx/` 那幾個模組清完之後**,不是現在。 +Protocol 一旦標上去,它們的內部型別錯誤就會一起浮出來。`linux_wayland` 已經清完, +**所以這條剩下的前置是 `windows/`(9 個模組)、`linux_with_x11/`(3)與 `osx/`(1)**, +清完才輪到它。 有一件事別再踩:**這個閘門的判定不能隨環境浮動**。裝了 `[gui]`/`[webrtc]` 的開發機 與乾淨的 `pip install -e .` 曾經對 38 個模組看法不同(36 個 Qt 模組只在 PySide6 diff --git a/WHATS_NEW.md b/WHATS_NEW.md index 1a37d371..84618130 100644 --- a/WHATS_NEW.md +++ b/WHATS_NEW.md @@ -108,6 +108,38 @@ which meant 其餘模組 counted it a second time. The row is now two plain numb `wrapper/window_backends/` row that section had been missing entirely, and 其餘模組 no longer double-counts 3,293 lines. +### The Wayland Cluster: Four Modules, One Invariant Nobody Had Written Down + +The next cluster off the typing list is `linux_wayland` — `libei`, `_detect`, +`capture`, `screen` — and thirty-three of its forty errors were one sentence +repeated: `Item "None" of "BoundSymbols | None" has no attribute ei_…`. + +`LibeiBackend` holds its resolved entry points as `Optional[BoundSymbols]` +(`None` on a host without libei) and read them straight off that attribute at +every call site. Every one of those sites is in fact reached only after a guard +— `connect()` refuses an unavailable backend, `_emit` refuses a disconnected +one — so nothing was broken here. But the guarantee lived in the call graph +rather than anywhere a reader or a checker could see it, and what a new call +site that skipped the guard would raise is `AttributeError`: not an +`AutoControlException`, and therefore straight through every containment +boundary in the framework. The entry points now come through one `_api` +property that raises `LibeiUnavailable` — which is what this module's own +docstring says every failure in it raises. `_teardown` is the deliberate +exception: it runs from an `except BaseException` handler, so it narrows the +attribute itself rather than risking a raise that would replace the real +failure with a complaint about the symbol table. + +The other three modules were each one honest disagreement. `_detect`'s two +environment probes were annotated `dict` while both of them default to +`os.environ`, which is a `Mapping` — so one of them could not legally pass its +own environment to the other. `capture._write_to_temp_png` declared a writer +returning `None` while every caller passes one returning the tool's stdout, +which it discards because what it reads is the file. And `screen.get_pixel` +handed Pillow's `getpixel` union — a float for mode `F`, `None` for an empty +band — to callers that unpack three ints, though `grab_image` has always +converted to RGB first. **141 modules left, 876 of 1,017 files inside the +contract.** + ## What's new (2026-08-20) ### Three Tests That Had Been Skipped Since They Were Written diff --git a/architecture_explore.md b/architecture_explore.md index b180f899..825ed026 100644 --- a/architecture_explore.md +++ b/architecture_explore.md @@ -20,7 +20,7 @@ iOS(WebDriverAgent)。核心能力是滑鼠/鍵盤控制、影像辨識、 | 指標 | 數值 | | --- | ---: | | Python 模組總數(含周邊子專案) | 1,031 | -| 程式碼總行數 | 140,388 | +| 程式碼總行數 | 140,422 | | `je_auto_control/utils/` 子套件數 | 310 | | `AC_*` 動作指令數(`known_commands()` 實測) | 773 | | 套件門面 `__all__` 公開名稱數 | 1,238 | @@ -229,7 +229,7 @@ socket server 有 8 MiB 讀取上限與 30 秒 handler timeout。 | `uinput/keyboard.py` | 33 | uinput 鍵盤後端,介面與 X11 版一致。 | | `uinput/mouse.py` | 116 | uinput 滑鼠後端。 | -#### Linux Wayland(`linux_wayland/`,17 檔/2,836 行) +#### Linux Wayland(`linux_wayland/`,17 檔/2,870 行) | 模組 | 行數 | 職責 | | --- | ---: | --- | @@ -240,13 +240,13 @@ socket server 有 8 MiB 讀取上限與 30 秒 handler timeout。 | `_select_input.py` | 85 | 決定使用原生 libei 或 CLI shim;`active_backend()` 是 keyboard/mouse 的唯一入口,`emitted()` 讓被拒絕的單次發送退回 CLI。 | | `_layout.py` | 83 | 版面原點的共用查詢。擷取與輸入不是同一個座標空間,差的就是這個原點:libei 的 region offset 是 `uint32`(描述不了負原點),`ydotool mousemove --absolute` 的原點是合成器夾取的那個角落——兩條路都要減掉它,所以放在這裡而不是各自複製。讀數快取一秒——擷取那一側刻意不快取,但 ydotool 每次絕對移動都會問,不快取等於每次移動多開一個 `wlr-randr` 行程。 | | `oeffis.py` | 196 | liboeffis 綁定:跑完 RemoteDesktop portal 交握,交出 EIS fd。 | -| `libei.py` | 611 | libei 綁定與完整握手(seat 綁定能力 → 由事件取得 device → start_emulating → 每次發送後 frame)。另負責絕對指標的座標空間:讀回裝置的 region,把版面座標映射進去,沒有任何 region 涵蓋就拒絕(libei 對這種移動是靜靜丟掉的)。 | +| `libei.py` | 632 | libei 綁定與完整握手(seat 綁定能力 → 由事件取得 device → start_emulating → 每次發送後 frame)。另負責絕對指標的座標空間:讀回裝置的 region,把版面座標映射進去,沒有任何 region 涵蓋就拒絕(libei 對這種移動是靜靜丟掉的)。 | | `mouse.py` | 384 | 滑鼠後端:移動、按鈕與捲動都 libei 優先,退回 ydotool;送往 libei 時垂直捲動軸取負(kernel `REL_WHEEL` 與 `wl_pointer` 正負號相反)。退到 ydotool 的絕對移動會先減掉版面原點(`--absolute` 是相對於版面左上角,不是版面座標的 `(0, 0)`),並依 `pointer_accel_mode()` 處理指標加速度——倍率讀不回來,只有操作者知道,所以由 `JE_AUTOCONTROL_WAYLAND_POINTER_ACCEL` 宣告:未設定=每個行程警告一次後照送、`flat`=已關掉加速度故靜靜送出、`strict`=拒絕這次移動。 | | `keyboard.py` | 173 | 鍵盤後端:libei 優先,退回 ydotool/wtype。 | | `keymap.py` | 155 | 友善鍵名 → evdev key code。 | -| `capture.py` | 236 | 擷取分層:操作者自訂指令 → grim → gnome-screenshot → spectacle → portal。 | +| `capture.py` | 241 | 擷取分層:操作者自訂指令 → grim → gnome-screenshot → spectacle → portal。 | | `portal.py` | 207 | `org.freedesktop.portal.Screenshot` 最後備援,經 `_dbus_client` 直接講 D-Bus(不再需要安裝 `gdbus`,只要有 session bus)。 | -| `screen.py` | 266 | 螢幕後端;發布 `grab_image` 與 `layout_origin`(擷取畫面左上角的版面座標,有螢幕在主螢幕左側/上方時為負),全框架的擷取都經由它。 | +| `screen.py` | 274 | 螢幕後端;發布 `grab_image` 與 `layout_origin`(擷取畫面左上角的版面座標,有螢幕在主螢幕左側/上方時為負),全框架的擷取都經由它。 | | `listener.py` / `record.py` | 48 / 34 | 監聽與錄製 stub(Wayland 限制)。 | #### 行動裝置 @@ -1036,7 +1036,7 @@ socket 預設綁 `127.0.0.1`;資源一律用 `with`。 | `utils/rest_api/` | 8 | 1,739 | | `utils/agent/` | 8 | 1,250 | | `linux_with_x11/` | 19 | 1,215 | -| `linux_wayland/` | 17 | 2,836 | +| `linux_wayland/` | 17 | 2,870 | | `utils/triggers/` | 4 | 1,146 | | `utils/ocr/` | 9 | 1,112 | | `utils/usbip/` | 5 | 920 | @@ -1045,5 +1045,5 @@ socket 預設綁 `127.0.0.1`;資源一律用 `with`。 | `autocontrol-lsp/` | 8 | 744 | | `utils/hotkey/` | 7 | 727 | | 其餘模組(約 286 個 `utils/` 子套件 + `android/`/`ios/`/周邊小工具) | 673 | 47,454 | -| **總計** | **1,025** | **140,323** | +| **總計** | **1,025** | **140,357** | diff --git a/je_auto_control/linux_wayland/_detect.py b/je_auto_control/linux_wayland/_detect.py index 98bfff4d..7f22e899 100644 --- a/je_auto_control/linux_wayland/_detect.py +++ b/je_auto_control/linux_wayland/_detect.py @@ -8,7 +8,7 @@ import os import shutil -from typing import Iterable, List, Optional +from typing import Iterable, List, Mapping, Optional WAYLAND_WTYPE = "wtype" @@ -33,7 +33,7 @@ def _normalise(value: Optional[str]) -> str: return (value or "").strip().lower() -def is_wayland_session(environ: Optional[dict] = None) -> bool: +def is_wayland_session(environ: Optional[Mapping[str, str]] = None) -> bool: """Return True when the live environment looks like Wayland. Honours an ``XDG_SESSION_TYPE`` of ``wayland`` and the @@ -45,7 +45,7 @@ def is_wayland_session(environ: Optional[dict] = None) -> bool: return bool(_normalise(env.get("WAYLAND_DISPLAY"))) -def select_display_server(environ: Optional[dict] = None) -> str: +def select_display_server(environ: Optional[Mapping[str, str]] = None) -> str: """Pick ``"wayland"`` or ``"x11"`` based on env + override. ``JE_AUTOCONTROL_LINUX_DISPLAY_SERVER=x11|wayland`` forces a diff --git a/je_auto_control/linux_wayland/capture.py b/je_auto_control/linux_wayland/capture.py index 9f454737..702f2af2 100644 --- a/je_auto_control/linux_wayland/capture.py +++ b/je_auto_control/linux_wayland/capture.py @@ -176,8 +176,13 @@ def _grim_capture(executable: str, return Capture(data, screen_region is not None) -def _write_to_temp_png(write: Callable[[str], None], label: str) -> Capture: - """Let ``write`` fill a temporary PNG, then read and delete it.""" +def _write_to_temp_png(write: Callable[[str], object], label: str) -> Capture: + """Let ``write`` fill a temporary PNG, then read and delete it. + + ``write`` may return anything — every caller wraps ``run_tool``, which + hands back the tool's stdout — because what is read here is the file, + not the return value. + """ handle, output_path = tempfile.mkstemp(prefix="je_autocontrol_", suffix=".png") os.close(handle) diff --git a/je_auto_control/linux_wayland/libei.py b/je_auto_control/linux_wayland/libei.py index ddf00cc0..7c866ca5 100644 --- a/je_auto_control/linux_wayland/libei.py +++ b/je_auto_control/linux_wayland/libei.py @@ -38,6 +38,7 @@ import select import threading import time +from functools import partial from typing import Callable, Dict, List, Optional, Sequence, Tuple from je_auto_control.linux_wayland import oeffis @@ -201,6 +202,22 @@ def is_connected(self) -> bool: """Whether the handshake finished and a device is emulating.""" return self._ei is not None and self._has_required_devices() + @property + def _api(self) -> BoundSymbols: + """The resolved entry points, or a refusal if libei never resolved. + + Every emission and handshake step reads its entry point through here + rather than off ``self._symbols`` directly. The two differ only on a + host without libei, and there the difference is the whole point: the + attribute is ``None``, so touching it raises ``AttributeError`` — + which is not an ``AutoControlException`` and therefore escapes every + containment boundary in the framework. This raises what the rest of + the module promises instead. + """ + if self._symbols is None: + raise LibeiUnavailable("libei.so.* not found on the loader path") + return self._symbols + def connect(self, *, timeout: float = HANDSHAKE_TIMEOUT, socket_path: Optional[bytes] = None) -> None: """Open a backend and run the handshake through to a live device. @@ -213,7 +230,7 @@ def connect(self, *, timeout: float = HANDSHAKE_TIMEOUT, with self._lock: if self.is_connected: return - sender = self._symbols.ei_new_sender(None) + sender = self._api.ei_new_sender(None) if not sender: raise LibeiUnavailable("ei_new_sender returned NULL") self._ei = sender @@ -237,13 +254,13 @@ def disconnect(self) -> None: def press_key(self, keycode: int) -> None: """Send a keydown for one evdev key code.""" self._emit(EI_DEVICE_CAP_KEYBOARD, lambda device: - self._symbols.ei_device_keyboard_key( + self._api.ei_device_keyboard_key( device, int(keycode), True)) def release_key(self, keycode: int) -> None: """Send a keyup for one evdev key code.""" self._emit(EI_DEVICE_CAP_KEYBOARD, lambda device: - self._symbols.ei_device_keyboard_key( + self._api.ei_device_keyboard_key( device, int(keycode), False)) def set_position(self, x: int, y: int) -> None: @@ -254,19 +271,19 @@ def set_position(self, x: int, y: int) -> None: is turned into a refusal instead of a silent no-op. """ self._emit(EI_DEVICE_CAP_POINTER_ABSOLUTE, lambda device: - self._symbols.ei_device_pointer_motion_absolute( + self._api.ei_device_pointer_motion_absolute( device, *self._region_point(device, x, y))) def press_button(self, button_code: int) -> None: """Press one BTN_* code (272 is BTN_LEFT).""" self._emit(EI_DEVICE_CAP_BUTTON, lambda device: - self._symbols.ei_device_button_button( + self._api.ei_device_button_button( device, int(button_code), True)) def release_button(self, button_code: int) -> None: """Release one BTN_* code.""" self._emit(EI_DEVICE_CAP_BUTTON, lambda device: - self._symbols.ei_device_button_button( + self._api.ei_device_button_button( device, int(button_code), False)) def click_button(self, button_code: int) -> None: @@ -284,7 +301,7 @@ def scroll(self, dx: int, dy: int) -> None: 1, did you mean 120?" bug warning showed up. """ self._emit(EI_DEVICE_CAP_SCROLL, lambda device: - self._symbols.ei_device_scroll_discrete( + self._api.ei_device_scroll_discrete( device, int(dx) * SCROLL_UNIT, int(dy) * SCROLL_UNIT)) # --- handshake -------------------------------------------------------- @@ -299,7 +316,7 @@ def _open_backend(self, socket_path: Optional[bytes]) -> None: if socket_path is None and not oeffis.is_available(): socket_path = _default_socket_path() if socket_path is not None: - code = self._symbols.ei_setup_backend_socket(self._ei, socket_path) + code = self._api.ei_setup_backend_socket(self._ei, socket_path) if code != 0: raise LibeiUnavailable( f"ei_setup_backend_socket returned {code}", @@ -311,7 +328,7 @@ def _open_backend(self, socket_path: Optional[bytes]) -> None: except oeffis.OeffisUnavailable as error: raise LibeiUnavailable(str(error)) from error self._session = session - code = self._symbols.ei_setup_backend_fd(self._ei, int(eis_fd)) + code = self._api.ei_setup_backend_fd(self._ei, int(eis_fd)) if code != 0: raise LibeiUnavailable(f"ei_setup_backend_fd returned {code}") self._backend_open = True @@ -329,34 +346,34 @@ def _handshake(self, deadline: float) -> None: def _pump(self, timeout: float) -> None: """Dispatch libei and answer every event that is waiting.""" - poll_fd = int(self._symbols.ei_get_fd(self._ei)) + poll_fd = int(self._api.ei_get_fd(self._ei)) if poll_fd < 0: raise LibeiUnavailable("ei_get_fd returned no pollable fd") if timeout > 0: ready, _, _ = select.select([poll_fd], [], [], timeout) if not ready: return - self._symbols.ei_dispatch(self._ei) + self._api.ei_dispatch(self._ei) while True: - event = self._symbols.ei_get_event(self._ei) + event = self._api.ei_get_event(self._ei) if not event: return try: self._on_event(event) finally: - self._symbols.ei_event_unref(event) + self._api.ei_event_unref(event) def _on_event(self, event: int) -> None: """Answer one libei event.""" - event_type = int(self._symbols.ei_event_get_type(event)) + event_type = int(self._api.ei_event_get_type(event)) if event_type == EI_EVENT_SEAT_ADDED: - self._bind_seat(self._symbols.ei_event_get_seat(event)) + self._bind_seat(self._api.ei_event_get_seat(event)) elif event_type == EI_EVENT_DEVICE_ADDED: - self._remember_device(self._symbols.ei_event_get_device(event)) + self._remember_device(self._api.ei_event_get_device(event)) elif event_type == EI_EVENT_DEVICE_RESUMED: - self._start_emulating(self._symbols.ei_event_get_device(event)) + self._start_emulating(self._api.ei_event_get_device(event)) elif event_type in (EI_EVENT_DEVICE_PAUSED, EI_EVENT_DEVICE_REMOVED): - self._forget_device(self._symbols.ei_event_get_device(event)) + self._forget_device(self._api.ei_event_get_device(event)) elif event_type == EI_EVENT_DISCONNECT: raise LibeiUnavailable("the compositor disconnected the sender") @@ -369,10 +386,11 @@ def _bind_seat(self, seat: int) -> None: """ if not seat: return - args = [ctypes.c_void_p(seat)] - args.extend(ctypes.c_int(cap) for cap in _WANTED_CAPS) - args.append(ctypes.c_void_p(None)) - self._symbols.ei_seat_bind_capabilities(*args) + self._api.ei_seat_bind_capabilities( + ctypes.c_void_p(seat), + *(ctypes.c_int(cap) for cap in _WANTED_CAPS), + ctypes.c_void_p(None), + ) def _remember_device(self, device: int) -> None: """Keep a reference to a device for each capability it carries.""" @@ -380,10 +398,10 @@ def _remember_device(self, device: int) -> None: return kept = False for cap in _WANTED_CAPS: - if not self._symbols.ei_device_has_capability(device, cap): + if not self._api.ei_device_has_capability(device, cap): continue if not kept: - self._symbols.ei_device_ref(device) + self._api.ei_device_ref(device) kept = True self._devices[cap] = device @@ -392,7 +410,7 @@ def _start_emulating(self, device: int) -> None: if not device or device not in self._devices.values(): return self._sequence += 1 - self._symbols.ei_device_start_emulating(device, self._sequence) + self._api.ei_device_start_emulating(device, self._sequence) self._emulating[device] = True def _forget_device(self, device: int) -> None: @@ -403,7 +421,7 @@ def _forget_device(self, device: int) -> None: stale = [cap for cap, known in self._devices.items() if known == device] for cap in stale: del self._devices[cap] - self._symbols.ei_device_unref(device) + self._api.ei_device_unref(device) def _has_required_devices(self) -> bool: return all(self._emulating.get(self._devices.get(cap, 0), False) @@ -422,14 +440,14 @@ def _device_regions(self, device: int) -> List[Region]: """ regions: List[Region] = [] for index in range(_MAX_REGIONS): - region = self._symbols.ei_device_get_region(device, index) + region = self._api.ei_device_get_region(device, index) if not region: break regions.append(( - int(self._symbols.ei_region_get_x(region)), - int(self._symbols.ei_region_get_y(region)), - int(self._symbols.ei_region_get_width(region)), - int(self._symbols.ei_region_get_height(region)), + int(self._api.ei_region_get_x(region)), + int(self._api.ei_region_get_y(region)), + int(self._api.ei_region_get_width(region)), + int(self._api.ei_region_get_height(region)), )) return regions @@ -487,7 +505,7 @@ def _emit(self, capability: int, send: Callable[[int], None]) -> None: f"no libei device is emulating capability {capability}", ) send(device) - self._symbols.ei_device_frame(device, self._symbols.ei_now(self._ei)) + self._api.ei_device_frame(device, self._api.ei_now(self._ei)) def _teardown(self) -> None: """Release what is safe to release; abandon what is not. @@ -520,11 +538,14 @@ def _teardown(self) -> None: process, since :func:`connected_backend` probes only once. A segfault in a library that drives someone's desktop is far worse than that. """ - if self._ei is not None and self._safe_to_unref(): + # Read through the attribute, not `_api`: this runs from the + # `except BaseException` handler in `connect`, where raising would + # replace the real failure with a complaint about the symbol table. + symbols = self._symbols + if symbols is not None and self._ei is not None and self._safe_to_unref(): for device in set(self._devices.values()): - _quietly(lambda handle=device: - self._symbols.ei_device_unref(handle)) - _quietly(lambda: self._symbols.ei_unref(self._ei)) + _quietly(partial(symbols.ei_device_unref, device)) + _quietly(partial(symbols.ei_unref, self._ei)) self._devices.clear() self._emulating.clear() self._ei = None diff --git a/je_auto_control/linux_wayland/screen.py b/je_auto_control/linux_wayland/screen.py index 9879514b..b31aac30 100644 --- a/je_auto_control/linux_wayland/screen.py +++ b/je_auto_control/linux_wayland/screen.py @@ -143,7 +143,15 @@ def get_pixel(x: int, y: int) -> Tuple[int, int, int]: :return: (R, G, B) """ image = grab_image([int(x), int(y), int(x) + 1, int(y) + 1]) - return image.getpixel((0, 0)) + pixel = image.getpixel((0, 0)) + # grab_image converts to RGB, so this is a 3-tuple; getpixel is + # annotated with the union of every mode's answer (a float for "F", + # None for an empty band), so say which one this is rather than + # handing the union to a caller that unpacks three ints. + if not isinstance(pixel, tuple) or len(pixel) < 3: + raise AutoControlScreenException( + f"wayland get_pixel: expected an RGB pixel, got {pixel!r}") + return int(pixel[0]), int(pixel[1]), int(pixel[2]) def screenshot(file_path: Optional[str] = None, diff --git a/test/verify/typing_contract_exempt.txt b/test/verify/typing_contract_exempt.txt index 87abee44..7ae186d3 100644 --- a/test/verify/typing_contract_exempt.txt +++ b/test/verify/typing_contract_exempt.txt @@ -14,7 +14,7 @@ # "Does not type-check" means on at least one of the three targets mypy can be # pointed at (win32, linux, darwin) — not just the one CI happens to run on. # -# Measured entries: 145 +# Measured entries: 141 je_auto_control.__main__ je_auto_control.cli @@ -32,10 +32,6 @@ je_auto_control.gui.recording_editor_tab je_auto_control.gui.remote_desktop.webrtc_dialogs je_auto_control.gui.remote_desktop.webrtc_panel je_auto_control.gui.test_suite_tab -je_auto_control.linux_wayland._detect -je_auto_control.linux_wayland.capture -je_auto_control.linux_wayland.libei -je_auto_control.linux_wayland.screen je_auto_control.linux_with_x11.listener.x11_linux_listener je_auto_control.linux_with_x11.record.x11_linux_record je_auto_control.linux_with_x11.uinput._device From c6075d76bd9499bcf050b285b226c8d608d42a64 Mon Sep 17 00:00:00 2001 From: JeffreyChen Date: Sat, 22 Aug 2026 04:20:25 +0800 Subject: [PATCH 06/19] Clear the x11 and macOS backends off the typing contract MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Five errors across four modules, one of them a live bug. The X11 listener's record_queue was assigned None with no annotation, so its type *was* None: record() could not fill it and stop_record() promised a Queue while able to return the None it started as. Stopping a recording that was never started reached x11_linux_record, which reads .queue off the result — an AttributeError a frame away from the mistake. It returns an empty queue now, matching what the public stop_record() already returns for the same case. osx_keyboard.press_key tested `keycode in special_key_table`, which narrows the string case into the branch but leaves int | str outside it, so an unknown name reached Quartz as a keycode. A string only ever names a special key here. uinput/_device's O_NONBLOCK is POSIX-only and the contract checks a Windows target too; the flag moved into a sys.platform branch mypy can prune. --- CHANGELOG.md | 7 +++++ Progress.md | 18 +++++++---- WHATS_NEW.md | 30 +++++++++++++++++++ architecture_explore.md | 12 ++++---- .../listener/x11_linux_listener.py | 12 +++++++- .../linux_with_x11/record/x11_linux_record.py | 2 +- .../linux_with_x11/uinput/_device.py | 13 +++++++- je_auto_control/osx/keyboard/osx_keyboard.py | 14 +++++++-- test/verify/typing_contract_exempt.txt | 6 +--- 9 files changed, 91 insertions(+), 23 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 19311d0f..8cf15be6 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -382,6 +382,13 @@ only when documented here with a migration path. unpacking through `_scroll_to` and now skips the pre-move instead, which is the graceful degradation its own comment already documented for backends that cannot report the cursor. +- **Stopping an X11 recording that was never started raised instead of + returning nothing.** The X11 listener's `stop_record()` handed back the + `None` its queue attribute was constructed with, and the recorder one frame + up reads `.queue` off that result, so `stop_record()` without a preceding + `record()` produced an `AttributeError` that the wrapper caught and logged + as a failure. It now returns an empty queue, so the public `stop_record()` + returns the empty list it documents. - **`check_key_is_press()` passed `None` to the backend for an unknown key name.** A name the virtual-key table has no entry for became `None` and was handed to the platform backend anyway: a `TypeError` on Windows and a silent diff --git a/Progress.md b/Progress.md index 5cb35b7e..1e8d1fbf 100644 --- a/Progress.md +++ b/Progress.md @@ -240,11 +240,11 @@ capability enum 值與 variadic `ei_seat_bind_capabilities`、event-type enum (`--cov-report=term-missing` 已經開著,CI 每一格都存了 `coverage.xml` artifact), 而不是齊頭式地補測試。 -### mypy:整包把關,141 個模組還沒過 +### mypy:整包把關,137 個模組還沒過 範圍不再是兩條路徑,而是**整包減去一張只准變少的清單** (`test/verify/typing_contract_exempt.txt`)。差別在於預設值:路徑清單只有人想到才會長, -新模組預設在圈外;現在新模組**預設就在契約裡**,1,017 個檔案有 876 個已經過關。 +新模組預設在圈外;現在新模組**預設就在契約裡**,1,017 個檔案有 880 個已經過關。 `wrapper` 那一群(6 個模組)在 2026-08-21 清掉了,做法見 [WHATS_NEW.md](WHATS_NEW.md): 平台縫的八個匯出名稱先宣告型別、再讓分支去綁定,其中三個用 @@ -260,7 +260,12 @@ capability enum 值與 variadic `ei_seat_bind_capabilities`、event-type enum `screen.get_pixel` 把 Pillow 那個「每種 mode 的答案聯集」收斂成它自己承諾的三元組。 做法見 [WHATS_NEW.md](WHATS_NEW.md)。 -剩下 141 個模組要清。錯誤碼分布是 `attr-defined` 佔大宗,其次 +`linux_with_x11`(3)與 `osx`(1)同日清掉:X11 listener 的 `record_queue` +從「型別是 None」變成 `Optional[Queue]`(`stop_record()` 因此不再回一個 +簽章不承認的 `None`),`uinput/_device` 的 POSIX 專屬 `O_NONBLOCK` 收進一個 +mypy 剪得掉的 `sys.platform` 分支,`osx_keyboard` 的字串 keycode 一律當特殊鍵名。 + +剩下 137 個模組要清。錯誤碼分布是 `attr-defined` 佔大宗,其次 `arg-type`/`union-attr`/`assignment`,成群集中在 `utils/remote_desktop`(13)、`gui`(11)、 `utils/accessibility/backends`/`utils/mcp_server`/`utils/usb/passthrough`(各 4)、 @@ -290,9 +295,10 @@ capability enum 值與 variadic `ei_seat_bind_capabilities`、event-type enum (`is_windows()`/`is_x11_unix()`),所以呼叫端這一側不必再改。 真正的成本在後端那一側:四個平台的 `keyboard`/`mouse` 模組本身都還在豁免清單上, -Protocol 一旦標上去,它們的內部型別錯誤就會一起浮出來。`linux_wayland` 已經清完, -**所以這條剩下的前置是 `windows/`(9 個模組)、`linux_with_x11/`(3)與 `osx/`(1)**, -清完才輪到它。 +Protocol 一旦標上去,它們的內部型別錯誤就會一起浮出來。`linux_wayland`、 +`linux_with_x11`、`osx` 都已經清完,**剩下的前置只有 `windows/`(9 個模組)**—— +其中 `win32_ctype_input`、`win32_ctype_mouse_control`、`win32_keypress_check` +與 `interception/` 那兩個正是 `keyboard`/`mouse` 在 Windows 上綁到的東西。 有一件事別再踩:**這個閘門的判定不能隨環境浮動**。裝了 `[gui]`/`[webrtc]` 的開發機 與乾淨的 `pip install -e .` 曾經對 38 個模組看法不同(36 個 Qt 模組只在 PySide6 diff --git a/WHATS_NEW.md b/WHATS_NEW.md index 84618130..5a1659da 100644 --- a/WHATS_NEW.md +++ b/WHATS_NEW.md @@ -140,6 +140,36 @@ band — to callers that unpack three ints, though `grab_image` has always converted to RGB first. **141 modules left, 876 of 1,017 files inside the contract.** +### The Three Backends That Only Needed Four Sentences + +`linux_with_x11` (3 modules) and `osx` (1) went the same day, and between them +they held five errors — but one was a live bug of a shape this branch has now +fixed several times. + +`KeypressHandler.record_queue` was assigned `None` in `__init__` with no +annotation, so its *type* was `None`: `record()` could not legally fill it, and +`stop_record()` promised a `Queue` while able to hand back the `None` it was +constructed with. Stopping a recording that was never started therefore reached +`x11_linux_record`, which reads `.queue` off the result — `AttributeError`, one +frame away from where the mistake was. It now returns an empty queue, which is +what "nothing was recorded" looks like and what `stop_record()` on the public +API already returns for the same case. + +`osx_keyboard.press_key` takes `int | str` and sends a string to +`special_key`. Testing `keycode in special_key_table` narrows the string case +*into* that branch but leaves `int | str` outside it — so a name the table does +not know fell through to `normal_key` and reached Quartz as a keycode. A string +only ever names a special key here, so that is now what the test asks, and an +unknown name gets `special_key`'s "Unknown special key" rather than a pyobjc +type error three frames down. + +The last one is a checker fact rather than a code fact: `uinput/_device` opens +`/dev/uinput` with the POSIX-only `O_NONBLOCK`, and the contract checks this +package against a Windows target too, where the `os` stub does not declare it. +The flag moved into a `sys.platform` branch mypy can prune, which states the +Linux-only-ness rather than silencing the question. **137 modules left, 880 of +1,017 files inside the contract.** + ## What's new (2026-08-20) ### Three Tests That Had Been Skipped Since They Were Written diff --git a/architecture_explore.md b/architecture_explore.md index 825ed026..bb3fe84c 100644 --- a/architecture_explore.md +++ b/architecture_explore.md @@ -20,7 +20,7 @@ iOS(WebDriverAgent)。核心能力是滑鼠/鍵盤控制、影像辨識、 | 指標 | 數值 | | --- | ---: | | Python 模組總數(含周邊子專案) | 1,031 | -| 程式碼總行數 | 140,422 | +| 程式碼總行數 | 140,451 | | `je_auto_control/utils/` 子套件數 | 310 | | `AC_*` 動作指令數(`known_commands()` 實測) | 773 | | 套件門面 `__all__` 公開名稱數 | 1,238 | @@ -201,7 +201,7 @@ socket server 有 8 MiB 讀取上限與 30 秒 handler timeout。 | `interception/keyboard.py` | 71 | 經 Interception 驅動的鍵盤輸入(繞過部分反自動化偵測)。 | | `interception/mouse.py` | 161 | 經 Interception 驅動的滑鼠輸入。 | -#### macOS(`osx/`,17 檔/907 行) +#### macOS(`osx/`,17 檔/915 行) | 模組 | 行數 | 職責 | | --- | ---: | --- | @@ -214,7 +214,7 @@ socket server 有 8 MiB 讀取上限與 30 秒 handler timeout。 | `screen/osx_screen.py` | 143 | 螢幕擷取與尺寸(含 Retina 座標處理)。 | | `pid/pid_control.py` | 64 | 以 PID 操作應用程式。 | -#### Linux X11(`linux_with_x11/`,19 檔/1,215 行) +#### Linux X11(`linux_with_x11/`,19 檔/1,236 行) | 模組 | 行數 | 職責 | | --- | ---: | --- | @@ -1035,15 +1035,15 @@ socket 預設綁 `127.0.0.1`;資源一律用 `with`。 | `windows/` | 23 | 1,900 | | `utils/rest_api/` | 8 | 1,739 | | `utils/agent/` | 8 | 1,250 | -| `linux_with_x11/` | 19 | 1,215 | +| `linux_with_x11/` | 19 | 1,236 | | `linux_wayland/` | 17 | 2,870 | | `utils/triggers/` | 4 | 1,146 | | `utils/ocr/` | 9 | 1,112 | | `utils/usbip/` | 5 | 920 | | `utils/assertion/` | 3 | 863 | -| `osx/` | 17 | 907 | +| `osx/` | 17 | 915 | | `autocontrol-lsp/` | 8 | 744 | | `utils/hotkey/` | 7 | 727 | | 其餘模組(約 286 個 `utils/` 子套件 + `android/`/`ios/`/周邊小工具) | 673 | 47,454 | -| **總計** | **1,025** | **140,357** | +| **總計** | **1,025** | **140,386** | diff --git a/je_auto_control/linux_with_x11/listener/x11_linux_listener.py b/je_auto_control/linux_with_x11/listener/x11_linux_listener.py index 9bf0b02c..14393554 100644 --- a/je_auto_control/linux_with_x11/listener/x11_linux_listener.py +++ b/je_auto_control/linux_with_x11/listener/x11_linux_listener.py @@ -1,5 +1,6 @@ from queue import Queue from threading import Thread +from typing import Optional from je_auto_control.utils.exception.exception_tags import linux_import_error_message, listener_error_message from je_auto_control.utils.exception.exceptions import AutoControlException @@ -41,7 +42,7 @@ def __init__(self, default_daemon: bool = True): self.daemon = default_daemon self.still_listener = True self.record_flag = False - self.record_queue = None + self.record_queue: Optional[Queue] = None self.event_keycode = 0 self.event_position = (0, 0) @@ -90,8 +91,17 @@ def stop_record(self) -> Queue: """ 停止記錄事件並回傳 Queue Stop recording and return the recorded queue + + 沒有呼叫過 ``record()`` 就停止時回傳空 Queue:原本回的是 ``None``, + 而簽章承諾的是 Queue,於是 ``x11_linux_record`` 在讀它的 ``.queue`` + 時拋 AttributeError。 + Returns an empty queue when ``record()`` was never called: it used to + hand back the ``None`` it was constructed with, and the caller one + frame up reads ``.queue`` off it. """ self.record_flag = False + if self.record_queue is None: + return Queue() return self.record_queue diff --git a/je_auto_control/linux_with_x11/record/x11_linux_record.py b/je_auto_control/linux_with_x11/record/x11_linux_record.py index 691bba00..5865f081 100644 --- a/je_auto_control/linux_with_x11/record/x11_linux_record.py +++ b/je_auto_control/linux_with_x11/record/x11_linux_record.py @@ -56,7 +56,7 @@ def stop_record(self) -> Queue[Any]: 停止錄製,並將結果轉換成動作序列 Queue """ self.result_queue = x11_linux_stop_record() - action_queue = Queue() + action_queue: Queue[Any] = Queue() # 將原始事件轉換成可讀格式 for details in self.result_queue.queue: diff --git a/je_auto_control/linux_with_x11/uinput/_device.py b/je_auto_control/linux_with_x11/uinput/_device.py index 54009847..6c2c80f6 100644 --- a/je_auto_control/linux_with_x11/uinput/_device.py +++ b/je_auto_control/linux_with_x11/uinput/_device.py @@ -12,6 +12,7 @@ import errno import os import struct +import sys import threading import time from typing import Optional @@ -21,6 +22,16 @@ # Layout cribbed from + . We only need # the subset that drives a standard keyboard + relative-mouse device. +if sys.platform == "win32": + # 只是為了通過型別檢查:`/dev/uinput` 只有 Linux 有,而型別契約會用三個 + # 目標平台各檢查一次,Windows 的 `os` stub 沒有 POSIX 專屬的 O_NONBLOCK。 + # Only has to type-check. /dev/uinput is Linux-only, and the typing + # contract checks this package against a Windows target too, where the + # `os` stub does not declare the POSIX-only O_NONBLOCK. + _OPEN_FLAGS = os.O_WRONLY +else: + _OPEN_FLAGS = os.O_WRONLY | os.O_NONBLOCK + _UINPUT_MAX_NAME_SIZE = 80 _BUS_USB = 0x03 @@ -119,7 +130,7 @@ def _pack_input_event(ev_type: int, code: int, value: int) -> bytes: def _open_device() -> int: """Open ``/dev/uinput`` and create the synthetic combo device.""" try: - fd = os.open("/dev/uinput", os.O_WRONLY | os.O_NONBLOCK) + fd = os.open("/dev/uinput", _OPEN_FLAGS) except OSError as exc: raise UinputUnavailable( "could not open /dev/uinput. Either load the module " diff --git a/je_auto_control/osx/keyboard/osx_keyboard.py b/je_auto_control/osx/keyboard/osx_keyboard.py index 38893233..a3670cc3 100644 --- a/je_auto_control/osx/keyboard/osx_keyboard.py +++ b/je_auto_control/osx/keyboard/osx_keyboard.py @@ -109,7 +109,11 @@ def press_key(keycode: int | str, is_shift: bool) -> None: :param keycode: 鍵盤代碼或特殊鍵名稱 :param is_shift: 是否同時按下 Shift """ - if keycode in special_key_table: + if isinstance(keycode, str): + # 字串在這裡只可能是特殊鍵名。表裡沒有的名字交給 `special_key` 說 + # 「不認識這個鍵」,而不是當成 keycode 丟給 Quartz。 + # A string only ever names a special key. One the table does not know + # is `special_key`'s refusal to make, not a keycode for Quartz. special_key(keycode, is_shift) else: normal_key(keycode, is_shift, True) @@ -123,7 +127,11 @@ def release_key(keycode: int | str, is_shift: bool) -> None: :param keycode: 鍵盤代碼或特殊鍵名稱 :param is_shift: 是否同時按下 Shift """ - if keycode in special_key_table: + if isinstance(keycode, str): + # 字串在這裡只可能是特殊鍵名。表裡沒有的名字交給 `special_key` 說 + # 「不認識這個鍵」,而不是當成 keycode 丟給 Quartz。 + # A string only ever names a special key. One the table does not know + # is `special_key`'s refusal to make, not a keycode for Quartz. special_key(keycode, is_shift) else: - normal_key(keycode, is_shift, False) \ No newline at end of file + normal_key(keycode, is_shift, False) diff --git a/test/verify/typing_contract_exempt.txt b/test/verify/typing_contract_exempt.txt index 7ae186d3..dd12da73 100644 --- a/test/verify/typing_contract_exempt.txt +++ b/test/verify/typing_contract_exempt.txt @@ -14,7 +14,7 @@ # "Does not type-check" means on at least one of the three targets mypy can be # pointed at (win32, linux, darwin) — not just the one CI happens to run on. # -# Measured entries: 141 +# Measured entries: 137 je_auto_control.__main__ je_auto_control.cli @@ -32,10 +32,6 @@ je_auto_control.gui.recording_editor_tab je_auto_control.gui.remote_desktop.webrtc_dialogs je_auto_control.gui.remote_desktop.webrtc_panel je_auto_control.gui.test_suite_tab -je_auto_control.linux_with_x11.listener.x11_linux_listener -je_auto_control.linux_with_x11.record.x11_linux_record -je_auto_control.linux_with_x11.uinput._device -je_auto_control.osx.keyboard.osx_keyboard je_auto_control.utils.accessibility.backends.base je_auto_control.utils.accessibility.backends.linux_backend je_auto_control.utils.accessibility.backends.windows_backend From 74027f843336785b7ceaaf848eaec90619fa8cd6 Mon Sep 17 00:00:00 2001 From: JeffreyChen Date: Sat, 22 Aug 2026 04:38:49 +0800 Subject: [PATCH 07/19] Fix what the typing gate found in the Windows backend MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Two of these are live defects rather than annotations. window_message did `from ...windows_window_manage import FindWindowW`, and that module exports no such name — FindWindowW is a method on its private user32 handle — so importing window_message raised ImportError on every Windows machine. Nothing caught it because the only importer in the tree is a manual test. It calls the public get_one_window_hwnd now, which is also the one that declares HWND-width argtypes. win32_ctype_input wrote ULONG_PTR into the standard library's own ctypes.wintypes namespace. The name appears exactly once in the tree, on that line, so the only thing the assignment could do was answer for another library in the same process. Its `_fields_: tuple` annotations also redeclared a ctypes ClassVar as an instance variable, and ctypes.POINTER and user32.SendInput were used as types when one is a function and the other a value. Every module under windows/ now type-checks on the win32 target. Eight stay exempt for a reason that is not about them, measured and left in Progress.md as a DECIDE: the gate also checks them as Linux and macOS, where typeshed does not declare the Win32-only corner of ctypes. --- CHANGELOG.md | 13 +++++ Progress.md | 44 ++++++++++++--- WHATS_NEW.md | 38 +++++++++++++ architecture_explore.md | 8 +-- .../windows/core/utils/win32_ctype_input.py | 51 +++++++++--------- .../windows/message/window_message.py | 8 +-- .../headless/test_windows_window_message.py | 54 +++++++++++++++++++ test/verify/typing_contract_exempt.txt | 3 +- 8 files changed, 178 insertions(+), 41 deletions(-) create mode 100644 test/unit_test/headless/test_windows_window_message.py diff --git a/CHANGELOG.md b/CHANGELOG.md index 8cf15be6..9b8d99e5 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -382,6 +382,19 @@ only when documented here with a migration path. unpacking through `_scroll_to` and now skips the pre-move instead, which is the graceful degradation its own comment already documented for backends that cannot report the cursor. +- **`je_auto_control.windows.message.window_message` could not be imported + at all.** It did `from ...windows_window_manage import FindWindowW`, and + that module has no such name — `FindWindowW` is a method on its private + `user32` handle — so importing `window_message` raised `ImportError` on + every Windows machine. It now calls the module's public + `get_one_window_hwnd`, which is also the one that declares HWND-width + argtypes rather than letting ctypes truncate a 64-bit handle to `c_int`. +- **Importing the Win32 input backend no longer writes into + `ctypes.wintypes`.** `win32_ctype_input` set `wintypes.ULONG_PTR = + wintypes.WPARAM` on the standard library's own module. Nothing in this + package ever read it back, so the only effect the assignment could have was + on some other library in the same process asking `ctypes.wintypes` whether + it has `ULONG_PTR`. - **Stopping an X11 recording that was never started raised instead of returning nothing.** The X11 listener's `stop_record()` handed back the `None` its queue attribute was constructed with, and the recorder one frame diff --git a/Progress.md b/Progress.md index 1e8d1fbf..f236f6dc 100644 --- a/Progress.md +++ b/Progress.md @@ -240,11 +240,11 @@ capability enum 值與 variadic `ei_seat_bind_capabilities`、event-type enum (`--cov-report=term-missing` 已經開著,CI 每一格都存了 `coverage.xml` artifact), 而不是齊頭式地補測試。 -### mypy:整包把關,137 個模組還沒過 +### mypy:整包把關,136 個模組還沒過 範圍不再是兩條路徑,而是**整包減去一張只准變少的清單** (`test/verify/typing_contract_exempt.txt`)。差別在於預設值:路徑清單只有人想到才會長, -新模組預設在圈外;現在新模組**預設就在契約裡**,1,017 個檔案有 880 個已經過關。 +新模組預設在圈外;現在新模組**預設就在契約裡**,1,017 個檔案有 881 個已經過關。 `wrapper` 那一群(6 個模組)在 2026-08-21 清掉了,做法見 [WHATS_NEW.md](WHATS_NEW.md): 平台縫的八個匯出名稱先宣告型別、再讓分支去綁定,其中三個用 @@ -265,7 +265,16 @@ capability enum 值與 variadic `ei_seat_bind_capabilities`、event-type enum 簽章不承認的 `None`),`uinput/_device` 的 POSIX 專屬 `O_NONBLOCK` 收進一個 mypy 剪得掉的 `sys.platform` 分支,`osx_keyboard` 的字串 keycode 一律當特殊鍵名。 -剩下 137 個模組要清。錯誤碼分布是 `attr-defined` 佔大宗,其次 +`windows/` 這一群的**真實錯誤全部清掉了**:`window_message` 匯入一個 +`windows_window_manage` 根本沒有匯出的名字(`FindWindowW`),所以那個模組在 +Windows 上一直是 import 就炸;`win32_ctype_input` 則有一批標錯的註記 +(`_fields_: tuple` 把 ClassVar 重新宣告成 instance variable、`ctypes.POINTER` +與 `user32.SendInput` 被當成型別用)以及一行改寫標準函式庫的 +`wintypes.ULONG_PTR = wintypes.WPARAM`——全樹沒有任何地方讀它。 +**以 `--platform win32` 檢查時,`je_auto_control/windows/` 底下已經零錯誤。** +剩下的豁免是下面那條 `DECIDE`。 + +剩下 136 個模組要清。錯誤碼分布是 `attr-defined` 佔大宗,其次 `arg-type`/`union-attr`/`assignment`,成群集中在 `utils/remote_desktop`(13)、`gui`(11)、 `utils/accessibility/backends`/`utils/mcp_server`/`utils/usb/passthrough`(各 4)、 @@ -296,9 +305,32 @@ mypy 剪得掉的 `sys.platform` 分支,`osx_keyboard` 的字串 keycode 一 真正的成本在後端那一側:四個平台的 `keyboard`/`mouse` 模組本身都還在豁免清單上, Protocol 一旦標上去,它們的內部型別錯誤就會一起浮出來。`linux_wayland`、 -`linux_with_x11`、`osx` 都已經清完,**剩下的前置只有 `windows/`(9 個模組)**—— -其中 `win32_ctype_input`、`win32_ctype_mouse_control`、`win32_keypress_check` -與 `interception/` 那兩個正是 `keyboard`/`mouse` 在 Windows 上綁到的東西。 +`linux_with_x11`、`osx` 都已經清完,而 `windows/` 的**真實**型別錯誤也清完了 +(見上),只剩下面那條 `DECIDE` 的 ctypes 表面問題。所以前置條件實質上已經到位, +可以開始標 Protocol;下一次動這條的人不必再等別的群集。 + +#### `DECIDE`:Windows 後端剩下的豁免不是它們的錯 + +`windows/` 底下還有 8 個模組留在清單上,而它們在 `--platform win32` 上**全部乾淨**。 +留在清單上的唯一原因是這個閘門會用三個目標平台各檢查一次,而 `ctypes` 的 +Win32 專屬表面(`windll`、`WinDLL`、`WINFUNCTYPE`、`WinError`、`get_last_error`) +在 typeshed 的 linux/darwin 目標上根本沒有宣告。共 18 處,分布在 8 個檔。 + +三條路都實測過,維護者挑一條: + +| 做法 | 成本 | 實測結果 | +| --- | --- | --- | +| **剪掉模組本體**(`if TYPE_CHECKING and sys.platform != "win32": raise`) | 每個檔 2 行 | **不能用**。mypy 一旦把模組本體判為 unreachable,`from ... import user32` 的那一端就會拿到 `Cannot determine type of "user32"` [has-type]——把 18 個錯誤換成一串新的、而且落在原本乾淨的模組上。 | +| **一個 shim 模組**把 Win32 專屬的 ctypes 表面集中匯出,非 Windows 分支綁 POSIX 對應物 | 新檔 + 18 個呼叫點改名 | 可行,事實只寫一次;代價是為了型別去動輸入熱路徑的 `user32` handle 建立方式。 | +| **18 個 `# type: ignore[attr-defined]`** 各附理由 | 18 行註記 | 可行,執行期零風險;代價是 18 個抑制,且同一句話重複 18 次。 | + +第四條是改閘門本身:讓 `typing_contract_verify.py` 只用模組自己的平台去量 +平台專屬模組(`windows/` 只用 win32、`osx/` 只用 darwin、`linux_*` 只用 linux)。 +這是三者裡最貼近事實的一條——拿 Linux 去檢查一個只跑得動在 Windows 的模組本來就沒有意義, +而閘門的 docstring 自己寫的多目標理由是「要讀得到平台分支後面的程式碼」,不是這個。 +但它會改掉「清單上的模組=在每個支援平台上都還沒過」這個既定語意,**所以要維護者拍板**。 + +推薦:第四條,其次是 shim。 有一件事別再踩:**這個閘門的判定不能隨環境浮動**。裝了 `[gui]`/`[webrtc]` 的開發機 與乾淨的 `pip install -e .` 曾經對 38 個模組看法不同(36 個 Qt 模組只在 PySide6 diff --git a/WHATS_NEW.md b/WHATS_NEW.md index 5a1659da..e2582950 100644 --- a/WHATS_NEW.md +++ b/WHATS_NEW.md @@ -170,6 +170,44 @@ The flag moved into a `sys.platform` branch mypy can prune, which states the Linux-only-ness rather than silencing the question. **137 modules left, 880 of 1,017 files inside the contract.** +### A Module Nobody Could Import, and a Line That Edited the Standard Library + +The Windows cluster turned out to hold the two most interesting findings on +this branch, and neither is a typing nicety. + +`je_auto_control.windows.message.window_message` did +`from ...windows_window_manage import FindWindowW`. That module has no such +name — `FindWindowW` is a method on its *private* `user32` handle — so +importing `window_message` raised `ImportError`, on every Windows machine, +since whenever the name was moved. Nothing noticed because the only importer +in the tree is a manual test. It now calls that module's public +`get_one_window_hwnd`, which is also the one carrying the argtypes that keep a +64-bit HWND from being truncated to `c_int`. + +`win32_ctype_input` ran `wintypes.ULONG_PTR = wintypes.WPARAM` at import — a +write into the standard library's own module namespace. Nothing in this package +reads `ULONG_PTR` back; measured, the name appears exactly once in the tree, on +that line. So the only thing the assignment could do was answer for some other +library in the same process that asked `ctypes.wintypes` whether it has +`ULONG_PTR`. Deleted. + +The same file also carried annotations that were wrong rather than merely +unhelpful: `_fields_: tuple` redeclares a ctypes `ClassVar` as an instance +variable, and `ctypes.POINTER` and `user32.SendInput` were used as types when +one is a function and the other a value. Dropping all three leaves exactly what +mypy infers, which was right all along. + +**Every module under `je_auto_control/windows/` now type-checks on the Windows +target.** Eight of them stay on the exemption list anyway, for a reason that is +not about them: the gate checks the package against Linux and macOS targets +too, where typeshed does not declare the Win32-only corner of `ctypes` +(`windll`, `WinDLL`, `WINFUNCTYPE`, `WinError`, `get_last_error`). Three +remedies were measured, one of them ruled out — pruning the module body makes +every *importer* fail with `has-type` instead — and `Progress.md` carries the +comparison as a `DECIDE`, because the cleanest of them changes what the gate +means rather than what the code says. **136 modules left, 881 of 1,017 files +inside the contract.** + ## What's new (2026-08-20) ### Three Tests That Had Been Skipped Since They Were Written diff --git a/architecture_explore.md b/architecture_explore.md index bb3fe84c..52b57515 100644 --- a/architecture_explore.md +++ b/architecture_explore.md @@ -20,7 +20,7 @@ iOS(WebDriverAgent)。核心能力是滑鼠/鍵盤控制、影像辨識、 | 指標 | 數值 | | --- | ---: | | Python 模組總數(含周邊子專案) | 1,031 | -| 程式碼總行數 | 140,451 | +| 程式碼總行數 | 140,452 | | `je_auto_control/utils/` 子套件數 | 310 | | `AC_*` 動作指令數(`known_commands()` 實測) | 773 | | 套件門面 `__all__` 公開名稱數 | 1,238 | @@ -183,7 +183,7 @@ socket server 有 8 MiB 讀取上限與 30 秒 handler timeout。 ### 5.3 平台後端 -#### Windows(`windows/`,23 檔/1,900 行) +#### Windows(`windows/`,23 檔/1,901 行) | 模組 | 行數 | 職責 | | --- | ---: | --- | @@ -1032,7 +1032,7 @@ socket 預設綁 `127.0.0.1`;資源一律用 `with`。 | `je_auto_control/`(頂層 3 檔) | 3 | 2,363 | | `utils/accessibility/` | 13 | 2,818 | | `wrapper/` | 19 | 3,293 | -| `windows/` | 23 | 1,900 | +| `windows/` | 23 | 1,901 | | `utils/rest_api/` | 8 | 1,739 | | `utils/agent/` | 8 | 1,250 | | `linux_with_x11/` | 19 | 1,236 | @@ -1045,5 +1045,5 @@ socket 預設綁 `127.0.0.1`;資源一律用 `with`。 | `autocontrol-lsp/` | 8 | 744 | | `utils/hotkey/` | 7 | 727 | | 其餘模組(約 286 個 `utils/` 子套件 + `android/`/`ios/`/周邊小工具) | 673 | 47,454 | -| **總計** | **1,025** | **140,386** | +| **總計** | **1,025** | **140,387** | diff --git a/je_auto_control/windows/core/utils/win32_ctype_input.py b/je_auto_control/windows/core/utils/win32_ctype_input.py index 6b41b597..ac292a8f 100644 --- a/je_auto_control/windows/core/utils/win32_ctype_input.py +++ b/je_auto_control/windows/core/utils/win32_ctype_input.py @@ -12,28 +12,26 @@ user32 = ctypes.WinDLL('user32', use_last_error=True) -wintypes.ULONG_PTR = wintypes.WPARAM - Mouse: int = 0 Keyboard: int = 1 Hardware: int = 2 class MouseInput(ctypes.Structure): - _fields_: tuple = (("dx", wintypes.LONG), - ("dy", wintypes.LONG), - ("mouseData", wintypes.DWORD), - ("dwFlags", wintypes.DWORD), - ("time", wintypes.DWORD), - ("dwExtraInfo", ctypes.c_void_p)) + _fields_ = (("dx", wintypes.LONG), + ("dy", wintypes.LONG), + ("mouseData", wintypes.DWORD), + ("dwFlags", wintypes.DWORD), + ("time", wintypes.DWORD), + ("dwExtraInfo", ctypes.c_void_p)) class KeyboardInput(ctypes.Structure): - _fields_: tuple = (("wVk", wintypes.WORD), - ("wScan", wintypes.WORD), - ("dwFlags", wintypes.DWORD), - ("time", wintypes.DWORD), - ("dwExtraInfo", ctypes.c_void_p)) + _fields_ = (("wVk", wintypes.WORD), + ("wScan", wintypes.WORD), + ("dwFlags", wintypes.DWORD), + ("time", wintypes.DWORD), + ("dwExtraInfo", ctypes.c_void_p)) def __init__(self, *args, **kwds): super(KeyboardInput, self).__init__(*args, **kwds) @@ -42,20 +40,20 @@ def __init__(self, *args, **kwds): class HardwareInput(ctypes.Structure): - _fields_: tuple = (("uMsg", wintypes.DWORD), - ("wParamL", wintypes.WORD), - ("wParamH", wintypes.WORD)) + _fields_ = (("uMsg", wintypes.DWORD), + ("wParamL", wintypes.WORD), + ("wParamH", wintypes.WORD)) class Input(ctypes.Structure): class INPUTUnion(ctypes.Union): - _fields_: tuple = (("ki", KeyboardInput), - ("mi", MouseInput), - ("hi", HardwareInput)) + _fields_ = (("ki", KeyboardInput), + ("mi", MouseInput), + ("hi", HardwareInput)) - _anonymous_: tuple = ("_input",) - _fields_: tuple = (("type", wintypes.DWORD), - ("_input", INPUTUnion)) + _anonymous_ = ("_input",) + _fields_ = (("type", wintypes.DWORD), + ("_input", INPUTUnion)) def _check_count(result, func, args) -> list: @@ -64,9 +62,12 @@ def _check_count(result, func, args) -> list: return args -LPINPUT: ctypes.POINTER = ctypes.POINTER(Input) +LPINPUT = ctypes.POINTER(Input) -SendInput: user32.SendInput = user32.SendInput +SendInput = user32.SendInput -user32.SendInput.errcheck = _check_count +# 回傳 args 原封不動是 ctypes 對 errcheck 的既定約定,而 typeshed 把這個 +# hook 標成回傳單一 _CData——任何 pass-through 的 errcheck 都滿足不了。 +# reason: returning args unchanged is ctypes' documented errcheck contract. +user32.SendInput.errcheck = _check_count # type: ignore[assignment] user32.SendInput.argtypes = (wintypes.UINT, ctypes.c_void_p, ctypes.c_int) diff --git a/je_auto_control/windows/message/window_message.py b/je_auto_control/windows/message/window_message.py index c2152b4e..53592213 100644 --- a/je_auto_control/windows/message/window_message.py +++ b/je_auto_control/windows/message/window_message.py @@ -1,7 +1,7 @@ from je_auto_control.utils.exception.exceptions import AutoControlException from je_auto_control.windows.core.utils.win32_ctype_input import user32 -from je_auto_control.windows.window.windows_window_manage import FindWindowW +from je_auto_control.windows.window.windows_window_manage import get_one_window_hwnd # Win32 API 函式指標 Win32 API function pointers PostMessageW = user32.PostMessageW @@ -57,7 +57,7 @@ def send_message_to_window(window_name: str, action_message: int, :param key_code_2: lParam :return: (HWND, 傳送狀態) """ - hwnd = FindWindowW(window_name) + hwnd = get_one_window_hwnd(None, window_name) if not hwnd: raise AutoControlException(f"Window '{window_name}' not found") post_status = SendMessageW(hwnd, action_message, key_code_1, key_code_2) @@ -80,7 +80,7 @@ def post_message_to_window(window_name: str, action_message: int, 使用 PostMessageW 對指定視窗名稱投遞訊息 Post message to a window by name using PostMessageW """ - hwnd = FindWindowW(window_name) + hwnd = get_one_window_hwnd(None, window_name) if not hwnd: raise AutoControlException(f"Window '{window_name}' not found") post_status = PostMessageW(hwnd, action_message, key_code_1, key_code_2) @@ -94,4 +94,4 @@ def post_message_to_window_hwnd(hwnd, action_message: int, Post message to a window by HWND using PostMessageW """ post_status = PostMessageW(hwnd, action_message, key_code_1, key_code_2) - return hwnd, post_status \ No newline at end of file + return hwnd, post_status diff --git a/test/unit_test/headless/test_windows_window_message.py b/test/unit_test/headless/test_windows_window_message.py new file mode 100644 index 00000000..77e91b5d --- /dev/null +++ b/test/unit_test/headless/test_windows_window_message.py @@ -0,0 +1,54 @@ +"""The Win32 message module has to be importable, and must not edit ctypes. + +Both of these were found by the typing contract rather than by anything that +ran: ``window_message`` imported a name its source module does not export, so +the module raised ``ImportError`` on every Windows machine and nothing in the +shipping path noticed because only a manual test imports it. And +``win32_ctype_input`` wrote ``ULONG_PTR`` into the standard library's own +``ctypes.wintypes`` namespace, which no code in this package ever read back — +so the only thing that assignment could do was answer for someone else's +``hasattr(wintypes, "ULONG_PTR")``. +""" +import ctypes +import sys +from ctypes import wintypes + +import pytest + +pytestmark = pytest.mark.skipif( + sys.platform not in ("win32", "cygwin", "msys"), + reason="Windows input backend modules only import on Windows.", +) + + +def test_window_message_module_imports(): + """It used to raise ImportError before any of its functions were reachable.""" + from je_auto_control.windows.message import window_message + + for name in ("send_message_to_window", "send_message_to_window_hwnd", + "post_message_to_window", "post_message_to_window_hwnd"): + assert callable(getattr(window_message, name)) + assert window_message.messages["WM_CLOSE"] == 0x0010 + + +def test_window_lookup_goes_through_the_prototyped_helper(): + """The helper it calls is the one that declares HWND-width argtypes. + + ``FindWindowW`` off a raw handle defaults to ``c_int``, which truncates a + 64-bit HWND; ``get_one_window_hwnd`` sets the prototype first. + """ + from je_auto_control.windows.message import window_message + from je_auto_control.windows.window import windows_window_manage + + assert window_message.get_one_window_hwnd is ( + windows_window_manage.get_one_window_hwnd) + missing = "je_auto_control window that does not exist 0f3a" + assert windows_window_manage.get_one_window_hwnd(None, missing) == 0 + + +def test_importing_the_input_module_leaves_ctypes_alone(): + """Importing a backend must not add names to a standard library module.""" + import je_auto_control.windows.core.utils.win32_ctype_input # noqa: F401 # reason: imported for its side effects, which is what this asserts about + + assert not hasattr(wintypes, "ULONG_PTR") + assert not hasattr(ctypes.wintypes, "ULONG_PTR") diff --git a/test/verify/typing_contract_exempt.txt b/test/verify/typing_contract_exempt.txt index dd12da73..846f2cf4 100644 --- a/test/verify/typing_contract_exempt.txt +++ b/test/verify/typing_contract_exempt.txt @@ -14,7 +14,7 @@ # "Does not type-check" means on at least one of the three targets mypy can be # pointed at (win32, linux, darwin) — not just the one CI happens to run on. # -# Measured entries: 137 +# Measured entries: 136 je_auto_control.__main__ je_auto_control.cli @@ -148,7 +148,6 @@ je_auto_control.windows.core.utils.win32_ctype_input je_auto_control.windows.core.utils.win32_keypress_check je_auto_control.windows.interception._dll je_auto_control.windows.interception.mouse -je_auto_control.windows.message.window_message je_auto_control.windows.mouse.win32_ctype_mouse_control je_auto_control.windows.record.win32_input_hook je_auto_control.windows.screen.win32_screen From ca08ffa1d3cb63e6a157cab5649654624b29ff55 Mon Sep 17 00:00:00 2001 From: JeffreyChen Date: Sat, 22 Aug 2026 16:17:51 +0800 Subject: [PATCH 08/19] Make the folder-sync tests wait for the baseline instead of guessing MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The macOS 3.14 grid cell failed on `sent.count("doc.txt") == 1` seeing 2: the test edited the file in place and then pushed its mtime forward, so between `write_text` and `os.utime` the file briefly carried a third, intermediate mtime. A tick landing in that window pushed the file once for the intermediate mtime and again for the final one — correct engine behaviour, an over-specified test. Stage the new content outside the watch dir and swap it in with `os.replace` so one edit is one event. The other half was the same guess in the other direction: every test slept a fixed 0.4s hoping the baseline snapshot had been taken, but the worker takes it asynchronously, so a slow runner could fold the test's own edit into the baseline and push nothing at all. `wait_until_ready()` makes that observable, and it is not test-only scaffolding — a caller that drops files right after `start()` has the same race. --- architecture_explore.md | 14 +++---- .../utils/remote_desktop/file_sync.py | 13 +++++++ test/unit_test/headless/test_folder_sync.py | 37 ++++++++++++------- 3 files changed, 43 insertions(+), 21 deletions(-) diff --git a/architecture_explore.md b/architecture_explore.md index 52b57515..87c643cd 100644 --- a/architecture_explore.md +++ b/architecture_explore.md @@ -20,7 +20,7 @@ iOS(WebDriverAgent)。核心能力是滑鼠/鍵盤控制、影像辨識、 | 指標 | 數值 | | --- | ---: | | Python 模組總數(含周邊子專案) | 1,031 | -| 程式碼總行數 | 140,452 | +| 程式碼總行數 | 140,465 | | `je_auto_control/utils/` 子套件數 | 310 | | `AC_*` 動作指令數(`known_commands()` 實測) | 773 | | 套件門面 `__all__` 公開名稱數 | 1,238 | @@ -510,14 +510,14 @@ socket server 有 8 MiB 讀取上限與 30 秒 handler timeout。 ### 5.4.10 遠端桌面與 USB -> 6 個套件、約 17,726 行。 +> 6 個套件、約 17,739 行。 | 模組 | 行數 | 職責 | | --- | ---: | --- | | `utils/admin/` | 327 | 多主機管理主控台:平行輪詢 N 個 AutoControl REST 端點 | | `utils/config_sync/` | 245 | 透過訊令伺服器做跨機器設定同步 | | `utils/device_matrix/` | 138 | 行動裝置矩陣:同一 action list 於多台裝置平行執行 | -| `utils/remote_desktop/` | 11,846 | **遠端桌面子系統**(56 檔/11.7K LOC):TCP/WebSocket/WebRTC 三條傳輸路徑、主機與檢視端、訊令伺服器、TURN/中繼、多檢視者、錄影、信任清單、TOTP、稽核鏈 | +| `utils/remote_desktop/` | 11,859 | **遠端桌面子系統**(56 檔/11.7K LOC):TCP/WebSocket/WebRTC 三條傳輸路徑、主機與檢視端、訊令伺服器、TURN/中繼、多檢視者、錄影、信任清單、TOTP、稽核鏈 | | `utils/usb/` | 4,250 | 跨平台 USB 列舉/熱插拔/裝置直通(WinUSB、IOKit、libusb 後端 + ACL + WebRTC DataChannel 通道) | | `utils/usbip/` | 920 | USB/IP 線路協定主機端(協定封包、TCP 伺服器、libusb URB 後端) | @@ -726,7 +726,7 @@ socket server 有 8 MiB 讀取上限與 30 秒 handler timeout。 | `rate_limit.py` | 48 | 工具呼叫的 token bucket 限流。 | | `__main__.py` | 87 | `je_auto_control_mcp` console script 進入點。 | -#### `utils/remote_desktop/`(11,846 行/56 檔) +#### `utils/remote_desktop/`(11,859 行/56 檔) 三條傳輸路徑並存:**TCP**(JPEG 影格)、**WebSocket**(同協定換傳輸)、**WebRTC**(aiortc 視訊 + DataChannel)。 @@ -768,7 +768,7 @@ socket server 有 8 MiB 讀取上限與 30 秒 handler timeout。 | `input_dispatch.py` | 133 | 在主機端套用輸入訊息。 | | `session_recorder.py` | 129 | 以 PyAV 把 WebRTC 影格錄成 mp4。 | | `totp.py` | 129 | RFC 6238 TOTP(零外部相依)。 | -| `file_sync.py` | 126 | 輪詢式資料夾鏡像。 | +| `file_sync.py` | 139 | 輪詢式資料夾鏡像。 | | `transport.py` | 123 | 可插拔的型別化訊息傳輸。 | | `host_access.py` | 105 | TCP 主機的檢視端核准與存取控制:`PendingViewer`、權限字串、分享碼的 TOTP 候選值、IP 白名單。`host` 與 `host_client` 共用,所以獨立成模組。 | | `protocol.py` | 96 | 長度前綴的 TCP 框架。 | @@ -1026,7 +1026,7 @@ socket 預設綁 `127.0.0.1`;資源一律用 `with`。 | --- | ---: | ---: | | `gui/` | 89 | 26,542 | | `utils/mcp_server/` | 21 | 17,323 | -| `utils/remote_desktop/` | 56 | 11,846 | +| `utils/remote_desktop/` | 56 | 11,859 | | `utils/executor/` | 6 | 9,075 | | `utils/usb/` | 17 | 4,250 | | `je_auto_control/`(頂層 3 檔) | 3 | 2,363 | @@ -1045,5 +1045,5 @@ socket 預設綁 `127.0.0.1`;資源一律用 `with`。 | `autocontrol-lsp/` | 8 | 744 | | `utils/hotkey/` | 7 | 727 | | 其餘模組(約 286 個 `utils/` 子套件 + `android/`/`ios/`/周邊小工具) | 673 | 47,454 | -| **總計** | **1,025** | **140,387** | +| **總計** | **1,025** | **140,400** | diff --git a/je_auto_control/utils/remote_desktop/file_sync.py b/je_auto_control/utils/remote_desktop/file_sync.py index 855ce51d..0079fa72 100644 --- a/je_auto_control/utils/remote_desktop/file_sync.py +++ b/je_auto_control/utils/remote_desktop/file_sync.py @@ -41,6 +41,7 @@ def __init__(self, *, watch_dir: Path, self._include_subdirs = bool(include_subdirs) self._snapshot: Dict[str, float] = {} # rel_path -> mtime self._stop = threading.Event() + self._ready = threading.Event() self._thread: Optional[threading.Thread] = None self._lifecycle_lock = threading.Lock() @@ -53,6 +54,7 @@ def start(self) -> None: f"watch dir not a directory: {self._watch}" ) self._stop.clear() + self._ready.clear() self._thread = threading.Thread( target=self._loop, name="folder-sync", daemon=True, ) @@ -72,6 +74,16 @@ def stop(self) -> None: def is_running(self) -> bool: return self._thread is not None and self._thread.is_alive() + def wait_until_ready(self, timeout: float = 5.0) -> bool: + """Block until the baseline snapshot exists; ``True`` if it does. + + ``start()`` returns as soon as the worker is spawned, so a file + created immediately after it can still land in the baseline and + never be pushed. Callers that add files right after starting + should wait here first. + """ + return self._ready.wait(timeout) + def _scan(self) -> Dict[str, float]: out: Dict[str, float] = {} try: @@ -94,6 +106,7 @@ def _loop(self) -> None: # as "already synced" so engaging sync mid-edit doesn't re-upload # the entire directory. self._snapshot = self._scan() + self._ready.set() while not self._stop.is_set(): self._stop.wait(self._interval) if self._stop.is_set(): diff --git a/test/unit_test/headless/test_folder_sync.py b/test/unit_test/headless/test_folder_sync.py index 02ca2a36..2e13d3f2 100644 --- a/test/unit_test/headless/test_folder_sync.py +++ b/test/unit_test/headless/test_folder_sync.py @@ -1,4 +1,5 @@ """Tests for FolderSyncEngine (round 22 — additive folder mirror).""" +import os import time import pytest @@ -8,7 +9,10 @@ @pytest.fixture() def watch_dir(tmp_path): - return tmp_path + """A dedicated watch dir, so a test can stage files beside it.""" + target = tmp_path / "watch" + target.mkdir() + return target def _make_engine(watch, sender, *, interval=0.2, include_subdirs=False): @@ -25,6 +29,7 @@ def test_pre_existing_files_not_pushed(watch_dir): engine = _make_engine(watch_dir, lambda p, n: sent.append(n)) engine.start() try: + assert engine.wait_until_ready() time.sleep(0.5) # one tick finally: engine.stop() @@ -36,31 +41,35 @@ def test_new_file_is_pushed(watch_dir): engine = _make_engine(watch_dir, lambda p, n: sent.append(n)) engine.start() try: - time.sleep(0.4) # let initial snapshot settle + assert engine.wait_until_ready() # baseline is fixed from here on (watch_dir / "new.txt").write_text("hi", encoding="utf-8") - time.sleep(0.6) + time.sleep(1.1) finally: engine.stop() assert "new.txt" in sent, sent -def test_modified_file_is_pushed_again(watch_dir): +def test_modified_file_is_pushed_again(watch_dir, tmp_path): sent = [] target = watch_dir / "doc.txt" target.write_text("v1", encoding="utf-8") engine = _make_engine(watch_dir, lambda p, n: sent.append(n)) engine.start() try: - time.sleep(0.4) - # bump mtime forward so the diff fires + assert engine.wait_until_ready() + # Bump mtime forward so the diff fires, but stage the new content + # *outside* the watch dir and swap it in atomically. Writing in + # place exposes an intermediate mtime between write and utime, and + # a tick landing in that window counts the edit twice. future = target.stat().st_mtime + 5.0 - target.write_text("v2", encoding="utf-8") - import os - os.utime(target, (future, future)) - time.sleep(0.6) + staged = tmp_path / "doc.txt.staged" + staged.write_text("v2", encoding="utf-8") + os.utime(staged, (future, future)) + os.replace(staged, target) + time.sleep(1.1) finally: engine.stop() - assert sent.count("doc.txt") == 1 + assert sent.count("doc.txt") == 1, sent def test_deletion_does_not_propagate(watch_dir): @@ -71,9 +80,9 @@ def test_deletion_does_not_propagate(watch_dir): engine = _make_engine(watch_dir, lambda p, n: sent.append(n)) engine.start() try: - time.sleep(0.4) + assert engine.wait_until_ready() target.unlink() - time.sleep(0.6) + time.sleep(1.1) finally: engine.stop() assert sent == [], f"deletion was propagated: {sent}" @@ -91,7 +100,7 @@ def flaky_sender(local_path, remote_name): engine = _make_engine(watch_dir, flaky_sender) engine.start() try: - time.sleep(0.7) + assert engine.wait_until_ready() (watch_dir / "retry.txt").write_text("data", encoding="utf-8") # Engine clamps interval to 0.5s minimum, so wait ≥1.5s for two ticks. time.sleep(1.7) From f9c1fc67abe7cccad4cd071238ee4acb1432af8f Mon Sep 17 00:00:00 2001 From: JeffreyChen Date: Sat, 22 Aug 2026 16:42:04 +0800 Subject: [PATCH 09/19] Clear the utils/remote_desktop cluster off the typing contract MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Thirteen modules, 169 errors, and only two shapes between them. Nine were `self._x = None` with no annotation — which makes the attribute's *type* `None`, so every later assignment is "incompatible types" and every later read is "None has no attribute". Several already carried the intended type in a trailing comment; those comments are now annotations, with the classes imported under TYPE_CHECKING so the lazy runtime imports stay lazy. Where the attribute is read back through a closure the receiver binds to a local first, because a closure re-reads the attribute and no narrowing survives that. The other four were mixins reading attributes they do not own. Each one's docstring already listed what it borrows from the host class it is mixed into; that list is now an `if TYPE_CHECKING:` block in the class body, which the runtime strips, so a declaration there cannot shadow what the host binds. Three of the fixes are behaviour rather than annotation: * `WebRTCLoopBridge._run` read the loop back off shared state after `start()` set it. The loop is passed to the thread instead, and `start()` returns it, so submit/call_soon hold what they use. * `_get_cursor_position` did `import sys as _sys` inside the function, which mypy does not recognise as a platform test — so its Win32 branch was checked against Linux and macOS as well. * `totp` caught `base64.binascii.Error`, a name that exists only because `base64` imports binascii itself. It imports binascii by name now. --- Progress.md | 24 +++++-- WHATS_NEW.md | 68 +++++++++++++++++++ architecture_explore.md | 30 ++++---- je_auto_control/utils/remote_desktop/audio.py | 6 +- .../utils/remote_desktop/host_capture.py | 19 +++++- .../utils/remote_desktop/host_service.py | 4 +- .../utils/remote_desktop/jpeg_recorder.py | 14 +++- .../utils/remote_desktop/session_recorder.py | 35 ++++++---- je_auto_control/utils/remote_desktop/totp.py | 3 +- .../utils/remote_desktop/video_codec.py | 27 ++++---- .../utils/remote_desktop/viewer.py | 4 +- .../utils/remote_desktop/webrtc_host.py | 61 +++++++++++------ .../utils/remote_desktop/webrtc_host_auth.py | 37 ++++++++-- .../utils/remote_desktop/webrtc_host_media.py | 22 ++++++ .../utils/remote_desktop/webrtc_transport.py | 45 +++++++----- .../utils/remote_desktop/webrtc_viewer.py | 62 +++++++++++------ test/verify/typing_contract_exempt.txt | 15 +--- 17 files changed, 338 insertions(+), 138 deletions(-) diff --git a/Progress.md b/Progress.md index f236f6dc..886fa86a 100644 --- a/Progress.md +++ b/Progress.md @@ -240,11 +240,11 @@ capability enum 值與 variadic `ei_seat_bind_capabilities`、event-type enum (`--cov-report=term-missing` 已經開著,CI 每一格都存了 `coverage.xml` artifact), 而不是齊頭式地補測試。 -### mypy:整包把關,136 個模組還沒過 +### mypy:整包把關,123 個模組還沒過 範圍不再是兩條路徑,而是**整包減去一張只准變少的清單** (`test/verify/typing_contract_exempt.txt`)。差別在於預設值:路徑清單只有人想到才會長, -新模組預設在圈外;現在新模組**預設就在契約裡**,1,017 個檔案有 881 個已經過關。 +新模組預設在圈外;現在新模組**預設就在契約裡**,1,017 個檔案有 894 個已經過關。 `wrapper` 那一群(6 個模組)在 2026-08-21 清掉了,做法見 [WHATS_NEW.md](WHATS_NEW.md): 平台縫的八個匯出名稱先宣告型別、再讓分支去綁定,其中三個用 @@ -274,13 +274,27 @@ Windows 上一直是 import 就炸;`win32_ctype_input` 則有一批標錯的 **以 `--platform win32` 檢查時,`je_auto_control/windows/` 底下已經零錯誤。** 剩下的豁免是下面那條 `DECIDE`。 -剩下 136 個模組要清。錯誤碼分布是 `attr-defined` 佔大宗,其次 +`utils/remote_desktop` 那一群(13 個模組、169 個錯誤,曾是最大的一群)在 +2026-08-22 清掉了,錯誤只有兩種形狀:九個模組是 `self._x = None` 沒有標注 +(mypy 就把那個屬性的**型別**判成 `None`),其中好幾個把想要的型別寫在行尾註解裡 +——事實一直都在,只是寫在沒有檢查器讀得到的地方;另外四個是 mixin 讀取 +自己不擁有的屬性,而每個 mixin 的 docstring 早就列出它跟宿主借了什麼, +現在那份清單變成類別本體裡的 `if TYPE_CHECKING:` 宣告(執行期會被剝掉, +所以不可能蓋掉宿主真正綁的東西)。順帶修掉三個行為問題: +`WebRTCLoopBridge._run` 從共用狀態讀 loop、`_get_cursor_position` 在函式內 +`import sys as _sys` 導致 mypy 剪不掉平台分支、`totp` 接的是 +`base64.binascii.Error`(那個名字只是 `base64` 自己 import 的副作用)。 +做法見 [WHATS_NEW.md](WHATS_NEW.md)。 + +剩下 123 個模組要清。錯誤碼分布是 `attr-defined` 佔大宗,其次 `arg-type`/`union-attr`/`assignment`,成群集中在 -`utils/remote_desktop`(13)、`gui`(11)、 +`gui`(11+`gui/remote_desktop` 2)、 `utils/accessibility/backends`/`utils/mcp_server`/`utils/usb/passthrough`(各 4)、 -`utils/observability`/`utils/triggers`(各 3)。 +`utils/observability`/`utils/triggers`(各 3),其餘散在各處。 **下一步**是一次清一個群集,清完把行從清單刪掉—— `python test/verify/typing_contract_verify.py --fix` 會替你改,CI 會在你忘了刪的時候紅掉。 +剛清掉的那一群留下一個可以直接套用的樣板:mixin 用 `if TYPE_CHECKING:` 宣告 +借來的成員,`Optional` 屬性用 `TYPE_CHECKING` 匯入真正的型別去標注。 #### 平台縫還缺的一半:`keyboard` 與 `mouse` 還沒有合約 diff --git a/WHATS_NEW.md b/WHATS_NEW.md index e2582950..9e076e34 100644 --- a/WHATS_NEW.md +++ b/WHATS_NEW.md @@ -208,6 +208,74 @@ comparison as a `DECIDE`, because the cleanest of them changes what the gate means rather than what the code says. **136 modules left, 881 of 1,017 files inside the contract.** +### The Biggest Remaining Cluster: Thirteen Modules Under `utils/remote_desktop` + +`Progress.md` named this one as the next step and as the largest group left +(13 modules, 169 errors). It came off in one pass, and the errors sorted into +exactly two shapes. + +**Nine of the thirteen were `self._x = None` with no annotation.** mypy infers +the attribute's *type* as `None` from that line, so every later assignment is +"incompatible types" and every later use is "None has no attribute …". Several +of them even carried the intended type in a trailing comment +(`self._files_receiver = None # Optional[FileTransferReceiver]`) — the fact was +known, just written somewhere no checker reads. Those comments are now +annotations, with the classes imported under `TYPE_CHECKING` so the lazy runtime +imports that keep startup cheap are untouched. Where an attribute is assigned +and then used through a closure — `_wire_files_channel` in both the host and the +viewer — the receiver is bound to a local first, because a closure re-reads the +attribute and no narrowing survives that. + +**The other four were mixins reading attributes they do not own.** +`MediaNegotiationMixin`, `ViewerAuthMixin` and `FrameProductionMixin` are halves +of a host class split for readability, and each one's docstring already listed +what it borrows from the class it is mixed into — `_pc`, `_config`, `_send_ctrl`, +`_spawn_bg`, `_shutdown`, `_clients`… That list is now a declaration: an +`if TYPE_CHECKING:` block in the class body naming each borrowed attribute and +method. The block is stripped at runtime, so a stub in it cannot shadow what the +host actually binds — which a plain class-body `def` would risk for any future +mixin sibling that does not define it. + +Three findings in the batch are behaviour, not annotation: + +- **`WebRTCLoopBridge._run` read the loop off shared state.** `start()` set + `self._loop` and spawned a thread whose target then read `self._loop` back to + run it. The loop is now passed to the thread as an argument, and `start()` + returns it, so `submit()` and `call_soon()` hand a value they hold rather than + re-reading an `Optional`. Same behaviour, one less cross-thread read. +- **`_get_cursor_position` was invisible to the platform pruner.** It did + `import sys as _sys` *inside* the function and branched on `_sys.platform`, + which mypy does not treat as a platform test — so the Win32 branch was + type-checked against Linux and macOS too. The import moved to module scope, + which is what makes `ctypes.windll` a Windows-only fact rather than an error. +- **`totp` caught `base64.binascii.Error`.** That attribute exists only because + `base64` imports `binascii` itself; nothing declares it, and it would vanish + with a stdlib refactor. `binascii` is now imported by name. + +Two dicts also stopped being dicts of `object`: `manifest.json`'s entry rows and +the four `BANDWIDTH_PRESETS` are `TypedDict`s, so `preset["fps"]` is an `int` +without a cast and the manifest's shape is stated where it is written rather +than inferred from three literals. **123 modules left, 894 of 1,017 files inside +the contract.** + +### The macOS Grid Cell That Failed on a Test's Own Race + +`pytest-headless (macos-14, 3.14)` went red on +`test_modified_file_is_pushed_again`, asserting one push and seeing two. The +engine was right and the test was not: it edited the watched file in place and +*then* pushed its mtime forward, so between `write_text` and `os.utime` the file +briefly carried a third, intermediate mtime. A poll tick landing in that window +legitimately pushes twice — once for the intermediate value and once for the +final one. The new content is now staged outside the watch dir and swapped in +with `os.replace`, so one edit is one event. + +The same file guessed in the other direction too: every test slept a fixed 0.4s +hoping the baseline snapshot had been taken, while `FolderSyncEngine.start()` +returns as soon as the worker thread is spawned. On a slow runner the test's own +edit could land *in* the baseline and be pushed never. `wait_until_ready()` makes +the handshake observable — and it is not test-only scaffolding: any caller that +drops files right after `start()` has exactly that race. + ## What's new (2026-08-20) ### Three Tests That Had Been Skipped Since They Were Written diff --git a/architecture_explore.md b/architecture_explore.md index 87c643cd..df3d3edd 100644 --- a/architecture_explore.md +++ b/architecture_explore.md @@ -20,7 +20,7 @@ iOS(WebDriverAgent)。核心能力是滑鼠/鍵盤控制、影像辨識、 | 指標 | 數值 | | --- | ---: | | Python 模組總數(含周邊子專案) | 1,031 | -| 程式碼總行數 | 140,465 | +| 程式碼總行數 | 140,596 | | `je_auto_control/utils/` 子套件數 | 310 | | `AC_*` 動作指令數(`known_commands()` 實測) | 773 | | 套件門面 `__all__` 公開名稱數 | 1,238 | @@ -510,14 +510,14 @@ socket server 有 8 MiB 讀取上限與 30 秒 handler timeout。 ### 5.4.10 遠端桌面與 USB -> 6 個套件、約 17,739 行。 +> 6 個套件、約 17,870 行。 | 模組 | 行數 | 職責 | | --- | ---: | --- | | `utils/admin/` | 327 | 多主機管理主控台:平行輪詢 N 個 AutoControl REST 端點 | | `utils/config_sync/` | 245 | 透過訊令伺服器做跨機器設定同步 | | `utils/device_matrix/` | 138 | 行動裝置矩陣:同一 action list 於多台裝置平行執行 | -| `utils/remote_desktop/` | 11,859 | **遠端桌面子系統**(56 檔/11.7K LOC):TCP/WebSocket/WebRTC 三條傳輸路徑、主機與檢視端、訊令伺服器、TURN/中繼、多檢視者、錄影、信任清單、TOTP、稽核鏈 | +| `utils/remote_desktop/` | 11,990 | **遠端桌面子系統**(56 檔/11.7K LOC):TCP/WebSocket/WebRTC 三條傳輸路徑、主機與檢視端、訊令伺服器、TURN/中繼、多檢視者、錄影、信任清單、TOTP、稽核鏈 | | `utils/usb/` | 4,250 | 跨平台 USB 列舉/熱插拔/裝置直通(WinUSB、IOKit、libusb 後端 + ACL + WebRTC DataChannel 通道) | | `utils/usbip/` | 920 | USB/IP 線路協定主機端(協定封包、TCP 伺服器、libusb URB 後端) | @@ -726,24 +726,24 @@ socket server 有 8 MiB 讀取上限與 30 秒 handler timeout。 | `rate_limit.py` | 48 | 工具呼叫的 token bucket 限流。 | | `__main__.py` | 87 | `je_auto_control_mcp` console script 進入點。 | -#### `utils/remote_desktop/`(11,859 行/56 檔) +#### `utils/remote_desktop/`(11,990 行/56 檔) 三條傳輸路徑並存:**TCP**(JPEG 影格)、**WebSocket**(同協定換傳輸)、**WebRTC**(aiortc 視訊 + DataChannel)。 | 檔案 | 行數 | 職責 | | --- | ---: | --- | -| `webrtc_host.py` | 683 | WebRTC 主機:串流螢幕視訊並接受檢視端輸入;session 生命週期、DataChannel 接線、檔案收發。 | -| `webrtc_viewer.py` | 638 | WebRTC 檢視端:接收視訊並送出輸入。 | +| `webrtc_host.py` | 702 | WebRTC 主機:串流螢幕視訊並接受檢視端輸入;session 生命週期、DataChannel 接線、檔案收發。 | +| `webrtc_viewer.py` | 662 | WebRTC 檢視端:接收視訊並送出輸入。 | | `host.py` | 625 | TCP 主機:接受迴圈、TLS 包裝、連線/認證握手、音訊與剪貼簿廣播、檔案推送、單次 token。 | | `viewer.py` | 623 | TCP 檢視端。 | | `host_service.py` | 542 | 無頭 WebRTC 主機執行器 + 多平台服務安裝器。 | | `host_client.py` | 406 | TCP 主機的每連線處理器:一個檢視端一個實例,擁有它的認證交換、sender/audio/receiver 三條執行緒,以及入站訊息的路由表。 | | `registry.py` | 370 | `AC_remote_*` 指令使用的行程級單例。 | -| `webrtc_transport.py` | 360 | 共用 WebRTC 管線:asyncio 橋接執行緒、螢幕視訊軌、設定。 | +| `webrtc_transport.py` | 369 | 共用 WebRTC 管線:asyncio 橋接執行緒、螢幕視訊軌、設定。 | | `multi_viewer.py` | 314 | 每個連入檢視端各跑一個 `WebRTCDesktopHost` 的協調器。 | | `signaling_server.py` | 297 | 獨立的 WebRTC SDP 交換 rendezvous 服務。 | | `audit_log.py` | 288 | SQLite 雜湊鏈稽核記錄。 | -| `host_capture.py` | 280 | TCP 主機的影格與游標產生:螢幕列舉、監視器索引轉擷取區域、預設 JPEG/游標 provider,以及 `FrameProductionMixin`(游標輪詢、擷取迴圈、上線編碼)。 | +| `host_capture.py` | 297 | TCP 主機的影格與游標產生:螢幕列舉、監視器索引轉擷取區域、預設 JPEG/游標 provider,以及 `FrameProductionMixin`(游標輪詢、擷取迴圈、上線編碼)。 | | `ws_protocol.py` | 277 | 最小 RFC 6455 WebSocket 框架與握手。 | | `file_transfer.py` | 273 | 分塊檔案傳輸。 | | `relay.py` | 270 | NAT 穿透失敗時的 TCP 中繼。 | @@ -754,10 +754,10 @@ socket server 有 8 MiB 讀取上限與 30 秒 handler timeout。 | `address_book.py` | 209 | 檢視端的主機通訊錄。 | | `audio.py` / `webrtc_audio.py` / `webrtc_mic.py` | 206 / 190 / 152 | 音訊擷取播放、音訊軌、麥克風上行。 | | `webrtc_files.py` | 205 | 專屬 DataChannel 的分塊檔案傳輸。 | -| `webrtc_host_auth.py` | 195 | 檢視端認證與核准:token 檢查、信任清單/IP 白名單自動放行、手動接受/拒絕、SAS、逾時關閉。 | +| `webrtc_host_auth.py` | 222 | 檢視端認證與核准:token 檢查、信任清單/IP 白名單自動放行、手動接受/拒絕、SAS、逾時關閉。 | | `lan_discovery.py` | 189 | mDNS/Zeroconf 區網探索。 | -| `video_codec.py` | 182 | TCP/WS 路徑的可插拔視訊編解碼。 | -| `webrtc_host_media.py` | 172 | 重新協商與 recvonly 軌管理。aiortc 沒有 `removeTransceiver`,所以開/關不對稱——開是加軌重新 offer,關只能設 inactive 並停掉 receiver。 | +| `video_codec.py` | 181 | TCP/WS 路徑的可插拔視訊編解碼。 | +| `webrtc_host_media.py` | 194 | 重新協商與 recvonly 軌管理。aiortc 沒有 `removeTransceiver`,所以開/關不對稱——開是加軌重新 offer,關只能設 inactive 並停掉 receiver。 | | `hw_codec.py` | 169 | 硬體 H.264 編碼偵測與啟用。 | | `webrtc_stats.py` | 163 | 把 aiortc 的 `RTCStats` 報告輪詢成精簡 dict。 | | `connect_coordinator.py` | 149 | 由使用者輸入的目標決定該用哪條傳輸。 | @@ -766,8 +766,8 @@ socket server 有 8 MiB 讀取上限與 30 秒 handler timeout。 | `trust_list.py` | 144 | 自動接受的檢視端信任清單。 | | `webrtc_inspector.py` | 138 | 行程級的 `StatsSnapshot` 滾動視窗。 | | `input_dispatch.py` | 133 | 在主機端套用輸入訊息。 | -| `session_recorder.py` | 129 | 以 PyAV 把 WebRTC 影格錄成 mp4。 | -| `totp.py` | 129 | RFC 6238 TOTP(零外部相依)。 | +| `session_recorder.py` | 134 | 以 PyAV 把 WebRTC 影格錄成 mp4。 | +| `totp.py` | 130 | RFC 6238 TOTP(零外部相依)。 | | `file_sync.py` | 139 | 輪詢式資料夾鏡像。 | | `transport.py` | 123 | 可插拔的型別化訊息傳輸。 | | `host_access.py` | 105 | TCP 主機的檢視端核准與存取控制:`PendingViewer`、權限字串、分享碼的 TOTP 候選值、IP 白名單。`host` 與 `host_client` 共用,所以獨立成模組。 | @@ -1026,7 +1026,7 @@ socket 預設綁 `127.0.0.1`;資源一律用 `with`。 | --- | ---: | ---: | | `gui/` | 89 | 26,542 | | `utils/mcp_server/` | 21 | 17,323 | -| `utils/remote_desktop/` | 56 | 11,859 | +| `utils/remote_desktop/` | 56 | 11,990 | | `utils/executor/` | 6 | 9,075 | | `utils/usb/` | 17 | 4,250 | | `je_auto_control/`(頂層 3 檔) | 3 | 2,363 | @@ -1045,5 +1045,5 @@ socket 預設綁 `127.0.0.1`;資源一律用 `with`。 | `autocontrol-lsp/` | 8 | 744 | | `utils/hotkey/` | 7 | 727 | | 其餘模組(約 286 個 `utils/` 子套件 + `android/`/`ios/`/周邊小工具) | 673 | 47,454 | -| **總計** | **1,025** | **140,400** | +| **總計** | **1,025** | **140,531** | diff --git a/je_auto_control/utils/remote_desktop/audio.py b/je_auto_control/utils/remote_desktop/audio.py index 5feb1ccc..5025bd59 100644 --- a/je_auto_control/utils/remote_desktop/audio.py +++ b/je_auto_control/utils/remote_desktop/audio.py @@ -12,7 +12,7 @@ """ import threading from dataclasses import dataclass -from typing import Callable, Optional +from typing import Any, Callable, Optional DEFAULT_SAMPLE_RATE = 16_000 DEFAULT_CHANNELS = 1 @@ -83,7 +83,7 @@ def __init__(self, on_block: AudioBlockCallback, self._sample_rate = int(sample_rate) self._channels = int(channels) self._block_frames = int(block_frames) - self._stream = None + self._stream: Optional[Any] = None self._lock = threading.Lock() @property @@ -153,7 +153,7 @@ def __init__(self, device: Optional[int] = None, self._device = device self._sample_rate = int(sample_rate) self._channels = int(channels) - self._stream = None + self._stream: Optional[Any] = None self._lock = threading.Lock() @property diff --git a/je_auto_control/utils/remote_desktop/host_capture.py b/je_auto_control/utils/remote_desktop/host_capture.py index 01ae4f1c..da2cdd04 100644 --- a/je_auto_control/utils/remote_desktop/host_capture.py +++ b/je_auto_control/utils/remote_desktop/host_capture.py @@ -8,6 +8,7 @@ still hosts. """ import json +import threading import time from io import BytesIO from typing import ( @@ -17,7 +18,7 @@ from je_auto_control.utils.logging.logging_instance import autocontrol_logger from je_auto_control.utils.remote_desktop.protocol import MessageType from je_auto_control.utils.remote_desktop.video_codec import ( - CODEC_JPEG, codec_tag, + CODEC_JPEG, CodecProvider, codec_tag, ) if TYPE_CHECKING: # avoids a runtime cycle: host_client is a sibling @@ -149,6 +150,22 @@ class FrameProductionMixin: ``_frame_lock``. """ + if TYPE_CHECKING: + # Declared, never defined: the host class this is mixed into owns + # every one of these. The block is stripped at runtime, so nothing + # here can shadow what the host actually binds. + _shutdown: threading.Event + _clients: List["_ClientHandler"] + _clients_lock: threading.Lock + _frame_cond: threading.Condition + _frame_provider: FrameProvider + _cursor_provider: Optional[CursorProvider] + _cursor_lock: threading.Lock + _latest_cursor_payload: Optional[bytes] + _codec_provider: CodecProvider + _latest_seq: int + _period: float + def _cursor_loop(self) -> None: """Poll cursor position at ~30 Hz and push it to viewers as JSON.""" provider = self._cursor_provider diff --git a/je_auto_control/utils/remote_desktop/host_service.py b/je_auto_control/utils/remote_desktop/host_service.py index 8b63b3fa..273dcc13 100644 --- a/je_auto_control/utils/remote_desktop/host_service.py +++ b/je_auto_control/utils/remote_desktop/host_service.py @@ -27,7 +27,7 @@ import time from dataclasses import dataclass from pathlib import Path -from typing import Optional +from typing import Any, Dict, Optional from je_auto_control.utils.logging.logging_instance import autocontrol_logger @@ -201,7 +201,7 @@ def _interactive_configure() -> int: """Prompt the user for the four required fields and write a config.""" print("AutoControl host service — interactive configuration") print(f"Config will be written to: {_DEFAULT_CONFIG_PATH}") - answers = {} + answers: Dict[str, Any] = {} answers["token"] = input("Auth token (shared with viewers): ").strip() answers["server_url"] = input("Signaling server URL: ").strip() answers["host_id"] = input("Host ID: ").strip() diff --git a/je_auto_control/utils/remote_desktop/jpeg_recorder.py b/je_auto_control/utils/remote_desktop/jpeg_recorder.py index 2b8695fd..a2ae920b 100644 --- a/je_auto_control/utils/remote_desktop/jpeg_recorder.py +++ b/je_auto_control/utils/remote_desktop/jpeg_recorder.py @@ -26,12 +26,20 @@ import threading import time from pathlib import Path -from typing import Dict, List, Optional +from typing import Dict, List, Optional, TypedDict _MANIFEST_FILENAME = "manifest.json" +class ManifestEntry(TypedDict): + """One row of ``manifest.json`` — what was written, when, how big.""" + + filename: str + timestamp: float + size: int + + class JpegSequenceRecorder: """Append-only JPEG-sequence recorder. @@ -47,7 +55,7 @@ def __init__(self, output_dir: str, *, self._prefix = file_prefix self._digits = max(1, int(digits)) self._lock = threading.Lock() - self._entries: List[Dict[str, float]] = [] + self._entries: List[ManifestEntry] = [] self._counter = 0 self._started = False self._stopped = False @@ -89,7 +97,7 @@ def record_frame(self, payload: bytes) -> None: filename = ( f"{self._prefix}_{self._counter:0{self._digits}d}.jpg" ) - entry = { + entry: ManifestEntry = { "filename": filename, "timestamp": time.time(), "size": len(payload), diff --git a/je_auto_control/utils/remote_desktop/session_recorder.py b/je_auto_control/utils/remote_desktop/session_recorder.py index b5208d3d..e60352a7 100644 --- a/je_auto_control/utils/remote_desktop/session_recorder.py +++ b/je_auto_control/utils/remote_desktop/session_recorder.py @@ -9,7 +9,7 @@ import threading from pathlib import Path -from typing import Optional +from typing import Optional, Tuple try: import av # type: ignore @@ -55,21 +55,26 @@ def __init__(self, output_path: str, *, self._started = False self._closed = False - def _open(self, frame) -> None: - if self._container is not None: - return + def _open(self, frame) -> Tuple[ + "av.container.OutputContainer", "av.video.stream.VideoStream", + ]: + """Return the open (container, stream) pair, creating it on first use.""" + if self._container is not None and self._stream is not None: + return self._container, self._stream self._path.parent.mkdir(parents=True, exist_ok=True) - self._container = av.open(str(self._path), mode="w") - stream = self._container.add_stream(self._codec, rate=self._fps) + container = av.open(str(self._path), mode="w") + stream = container.add_stream(self._codec, rate=self._fps) stream.width = frame.width stream.height = frame.height stream.pix_fmt = self._pixel_format + self._container = container self._stream = stream self._started = True autocontrol_logger.info( "session_recorder: writing to %s (%dx%d @%dfps, %s)", self._path, frame.width, frame.height, self._fps, self._codec, ) + return container, stream def write_frame(self, frame) -> None: """Encode one ``av.VideoFrame``; lazy-init the container.""" @@ -79,10 +84,9 @@ def write_frame(self, frame) -> None: if self._closed: return try: - self._open(frame) - packets = self._stream.encode(frame) - for packet in packets: - self._container.mux(packet) + container, stream = self._open(frame) + for packet in stream.encode(frame): + container.mux(packet) except (ValueError, OSError, RuntimeError) as error: autocontrol_logger.warning( "session_recorder: write failed, stopping: %r", error, @@ -99,17 +103,18 @@ def stop(self) -> None: self._teardown_locked() def _teardown_locked(self) -> None: - if self._stream is not None: + container, stream = self._container, self._stream + if stream is not None and container is not None: try: - for packet in self._stream.encode(None): - self._container.mux(packet) + for packet in stream.encode(None): + container.mux(packet) except (ValueError, OSError, RuntimeError) as error: autocontrol_logger.debug( "session_recorder: flush failed: %r", error, ) - if self._container is not None: + if container is not None: try: - self._container.close() + container.close() except (ValueError, OSError, RuntimeError) as error: autocontrol_logger.debug( "session_recorder: close failed: %r", error, diff --git a/je_auto_control/utils/remote_desktop/totp.py b/je_auto_control/utils/remote_desktop/totp.py index 07dd0c6d..f8e0ff7f 100644 --- a/je_auto_control/utils/remote_desktop/totp.py +++ b/je_auto_control/utils/remote_desktop/totp.py @@ -16,6 +16,7 @@ from __future__ import annotations import base64 +import binascii import hashlib import hmac import secrets @@ -48,7 +49,7 @@ def _decode_secret(secret: str) -> bytes: padding = "=" * ((8 - len(cleaned) % 8) % 8) try: return base64.b32decode(cleaned + padding, casefold=True) - except (ValueError, base64.binascii.Error) as exc: + except (ValueError, binascii.Error) as exc: raise TOTPError(f"invalid base32 secret: {exc}") from exc diff --git a/je_auto_control/utils/remote_desktop/video_codec.py b/je_auto_control/utils/remote_desktop/video_codec.py index 16547093..363b6cb0 100644 --- a/je_auto_control/utils/remote_desktop/video_codec.py +++ b/je_auto_control/utils/remote_desktop/video_codec.py @@ -20,7 +20,7 @@ """ from __future__ import annotations -from typing import Iterable, Optional +from typing import Any, Iterable, Optional CODEC_JPEG = "jpeg" CODEC_H264 = "h264" @@ -104,13 +104,14 @@ def __init__(self, self._width = width self._height = height self._gop_size = int(gop_size) - self._container = None - self._stream = None + self._container: Optional[Any] = None + self._stream: Optional[Any] = None self._closed = False - def _ensure_stream(self, width: int, height: int) -> None: + def _ensure_stream(self, width: int, height: int) -> Any: + """Return the encoder stream, opening the container on first use.""" if self._stream is not None: - return + return self._stream import av import io self._buffer = io.BytesIO() @@ -129,6 +130,7 @@ def _ensure_stream(self, width: int, height: int) -> None: "g": str(self._gop_size), } self._stream = stream + return stream def encode_jpeg(self, jpeg_bytes: bytes) -> Iterable[bytes]: if self._closed or not jpeg_bytes: @@ -136,16 +138,13 @@ def encode_jpeg(self, jpeg_bytes: bytes) -> Iterable[bytes]: import av # noqa: F401 lazy keep from io import BytesIO from PIL import Image - img = Image.open(BytesIO(jpeg_bytes)) - if img.mode != "RGB": - img = img.convert("RGB") - self._ensure_stream(img.width, img.height) - frame = av.VideoFrame.from_image(img) + image: Image.Image = Image.open(BytesIO(jpeg_bytes)) + if image.mode != "RGB": + image = image.convert("RGB") + stream = self._ensure_stream(image.width, image.height) + frame = av.VideoFrame.from_image(image) frame.pts = None - packets = [] - for packet in self._stream.encode(frame): - packets.append(bytes(packet)) - return packets + return [bytes(packet) for packet in stream.encode(frame)] def close(self) -> None: if self._closed: diff --git a/je_auto_control/utils/remote_desktop/viewer.py b/je_auto_control/utils/remote_desktop/viewer.py index 53ad7a32..f6b9ddec 100644 --- a/je_auto_control/utils/remote_desktop/viewer.py +++ b/je_auto_control/utils/remote_desktop/viewer.py @@ -225,7 +225,7 @@ def _maybe_wrap_tls(self, raw_sock: socket.socket) -> socket.socket: """Return a TLS-wrapped socket when an ssl_context was configured.""" if self._ssl_context is None: return raw_sock - hostname = self._server_hostname or self._host + hostname: Optional[str] = self._server_hostname or self._host if (self._ssl_context.check_hostname is False and self._ssl_context.verify_mode == ssl.CERT_NONE): # ``wrap_socket`` rejects server_hostname when verification is off. @@ -598,7 +598,7 @@ def _on_recv_cursor(self, payload: bytes, "remote_desktop viewer on_cursor callback raised" ) - def _notify_error(self, error: BaseException) -> None: + def _notify_error(self, error: Exception) -> None: if self._shutdown.is_set() or self._on_error is None: return try: diff --git a/je_auto_control/utils/remote_desktop/webrtc_host.py b/je_auto_control/utils/remote_desktop/webrtc_host.py index fa8670d9..2d57e59d 100644 --- a/je_auto_control/utils/remote_desktop/webrtc_host.py +++ b/je_auto_control/utils/remote_desktop/webrtc_host.py @@ -17,7 +17,7 @@ import asyncio import json import threading -from typing import Any, Callable, Mapping, Optional +from typing import TYPE_CHECKING, Any, Callable, Mapping, Optional from je_auto_control.utils.logging.logging_instance import autocontrol_logger from je_auto_control.utils.remote_desktop.audit_log import default_audit_log @@ -38,6 +38,18 @@ MediaNegotiationMixin, ) +if TYPE_CHECKING: # imported lazily at runtime to keep startup cheap + from je_auto_control.utils.remote_desktop.webrtc_audio import ( + OpusMicAudioTrack, OpusMicReceiver, + ) + from je_auto_control.utils.remote_desktop.webrtc_files import ( + FileTransferReceiver, + ) + from je_auto_control.utils.remote_desktop.webrtc_mic import ( + MicUplinkReceiver, + ) + from je_auto_control.utils.usb.passthrough import UsbChannelHost + _AUTH_GRACE_S = 5.0 _OFFER_TIMEOUT_S = 12.0 @@ -95,21 +107,22 @@ def __init__(self, *, token: str, # NOSONAR python:S107 # public constructor; self._pending_viewer_id: Optional[str] = None self._pc: Optional[RTCPeerConnection] = None self._video_track: Optional[ScreenVideoTrack] = None - self._control_channel = None - self._mic_channel = None - self._mic_receiver = None # Optional[MicUplinkReceiver] - self._files_channel = None - self._files_receiver = None # Optional[FileTransferReceiver] - self._usb_channel = None - self._usb_host = None # Optional[UsbChannelHost] + self._control_channel: Any = None + self._mic_channel: Any = None + self._mic_receiver: Optional[MicUplinkReceiver] = None + self._files_channel: Any = None + self._files_receiver: Optional[FileTransferReceiver] = None + self._usb_channel: Any = None + self._usb_host: Optional[UsbChannelHost] = None self._on_file_received: Optional[Callable] = None self._on_viewer_video_frame: Optional[Callable] = None - self._viewer_video_task = None - self._opus_audio_receiver = None # Optional[OpusMicReceiver] - self._host_voice_track = None # Optional[OpusMicAudioTrack] (outbound) + self._viewer_video_task: Optional[asyncio.Task] = None + self._opus_audio_receiver: Optional[OpusMicReceiver] = None + # Outbound host voice, when the host shares its own microphone. + self._host_voice_track: Optional[OpusMicAudioTrack] = None self._authenticated = False self._has_pending_viewer = False - self._auth_deadline_handle = None + self._auth_deadline_handle: Optional[asyncio.TimerHandle] = None # Hold strong refs to fire-and-forget tasks so the asyncio event # loop doesn't garbage-collect them mid-flight (S7502). Tasks # remove themselves from this set in their done callback. @@ -274,11 +287,12 @@ def _start_opus_audio_receive(self, track) -> None: if self._opus_audio_receiver is not None: return try: - self._opus_audio_receiver = OpusMicReceiver() + receiver = OpusMicReceiver() except (RuntimeError, OSError) as error: autocontrol_logger.warning("opus audio receiver init: %r", error) return - self._opus_audio_receiver.consume(track) + self._opus_audio_receiver = receiver + receiver.consume(track) autocontrol_logger.info("webrtc host: receiving Opus audio from viewer") async def _consume_viewer_video(self, track) -> None: @@ -362,8 +376,10 @@ def _wire_files_channel(self, channel) -> None: from je_auto_control.utils.remote_desktop.webrtc_files import ( FileTransferReceiver, ) - if self._files_receiver is None: - self._files_receiver = FileTransferReceiver(inbox_dir=self._inbox_dir) + receiver = self._files_receiver + if receiver is None: + receiver = FileTransferReceiver(inbox_dir=self._inbox_dir) + self._files_receiver = receiver @channel.on("message") def _on_message(message) -> None: @@ -376,7 +392,7 @@ def _on_message(message) -> None: if self._rate_limiter.should_warn_files(): self._safe_audit_log("rate_limit_files") return - self._files_receiver.handle_message( + receiver.handle_message( message, on_done=self._on_file_done, ) @@ -562,13 +578,16 @@ async def _async_apply_renegotiate_answer(self, sdp: str) -> None: self._maybe_resubscribe_viewer_video() self._maybe_resubscribe_viewer_audio() - def _ensure_files_receiver(self): + def _ensure_files_receiver(self) -> "FileTransferReceiver": + """Return the inbox receiver, creating it on first use.""" from je_auto_control.utils.remote_desktop.webrtc_files import ( FileTransferReceiver, ) - if self._files_receiver is None: - self._files_receiver = FileTransferReceiver(inbox_dir=self._inbox_dir) - return self._files_receiver + receiver = self._files_receiver + if receiver is None: + receiver = FileTransferReceiver(inbox_dir=self._inbox_dir) + self._files_receiver = receiver + return receiver def _handle_list_inbox(self) -> None: if not self._permissions.allow_files: diff --git a/je_auto_control/utils/remote_desktop/webrtc_host_auth.py b/je_auto_control/utils/remote_desktop/webrtc_host_auth.py index 434501e9..5b803532 100644 --- a/je_auto_control/utils/remote_desktop/webrtc_host_auth.py +++ b/je_auto_control/utils/remote_desktop/webrtc_host_auth.py @@ -10,7 +10,9 @@ from __future__ import annotations import asyncio -from typing import Any, Mapping, Optional +from typing import ( + TYPE_CHECKING, Any, Callable, Coroutine, List, Mapping, Optional, +) from je_auto_control.utils.logging.logging_instance import autocontrol_logger from je_auto_control.utils.remote_desktop.audit_log import default_audit_log @@ -19,6 +21,12 @@ ) from je_auto_control.utils.remote_desktop.webrtc_transport import get_bridge +if TYPE_CHECKING: # imported lazily at runtime to keep startup cheap + from je_auto_control.utils.remote_desktop.permissions import ( + SessionPermissions, + ) + from je_auto_control.utils.remote_desktop.trust_list import TrustList + class ViewerAuthMixin: """Auth half of :class:`WebRTCDesktopHost`. @@ -28,6 +36,23 @@ class ViewerAuthMixin: ``_send_ctrl``, ``_spawn_bg`` and ``_async_stop``. """ + if TYPE_CHECKING: + # Declared, never defined: the host class this is mixed into owns + # every one of these. The block is stripped at runtime, so nothing + # here can shadow what the host actually binds. + _token: str + _trust_list: Optional["TrustList"] + _ip_whitelist: List[str] + _permissions: "SessionPermissions" + _remote_ip: Optional[str] + _authenticated: bool + _auth_deadline_handle: Optional[asyncio.TimerHandle] + _on_authenticated: Optional[Callable[[], None]] + _on_pending_viewer: Optional[Callable[[], None]] + _send_ctrl: Callable[[Mapping[str, Any]], None] + _spawn_bg: Callable[[Any], asyncio.Task] + _async_stop: Callable[[], Coroutine[Any, Any, None]] + def _handle_send_sas(self) -> None: try: from je_auto_control.utils.remote_desktop.session_actions import ( @@ -74,15 +99,17 @@ def _reject_auth(self, data: Mapping[str, Any]) -> None: get_bridge().call_soon(self._schedule_close_after_fail) def _auto_approve_via_trust(self) -> bool: - if not self._is_trusted_viewer(self._pending_viewer_id): + # The emptiness check is what `_is_trusted_viewer` does first anyway; + # hoisting it makes the non-empty id available to `touch` below. + viewer_id = self._pending_viewer_id + if not viewer_id or not self._is_trusted_viewer(viewer_id): return False autocontrol_logger.info( - "webrtc host: viewer_id %s is trusted; auto-approving", - self._pending_viewer_id, + "webrtc host: viewer_id %s is trusted; auto-approving", viewer_id, ) if self._trust_list is not None: try: - self._trust_list.touch(self._pending_viewer_id) + self._trust_list.touch(viewer_id) except (RuntimeError, OSError) as error: autocontrol_logger.debug("trust touch: %r", error) self._approve_pending_viewer() diff --git a/je_auto_control/utils/remote_desktop/webrtc_host_media.py b/je_auto_control/utils/remote_desktop/webrtc_host_media.py index 2921f3b3..8dbf5bd0 100644 --- a/je_auto_control/utils/remote_desktop/webrtc_host_media.py +++ b/je_auto_control/utils/remote_desktop/webrtc_host_media.py @@ -8,11 +8,20 @@ """ from __future__ import annotations +import asyncio +from typing import TYPE_CHECKING, Any, Callable, Coroutine, Mapping, Optional + from je_auto_control.utils.logging.logging_instance import autocontrol_logger from je_auto_control.utils.remote_desktop.webrtc_transport import ( get_bridge, wait_for_ice_gathering, ) +if TYPE_CHECKING: # imported lazily at runtime to keep startup cheap + from je_auto_control.utils.remote_desktop.webrtc_audio import OpusMicReceiver + from je_auto_control.utils.remote_desktop.webrtc_transport import ( + RTCPeerConnection, WebRTCConfig, + ) + class MediaNegotiationMixin: """Media-track half of :class:`WebRTCDesktopHost`. @@ -22,6 +31,19 @@ class MediaNegotiationMixin: ``_consume_viewer_video`` and ``_start_opus_audio_receive``. """ + if TYPE_CHECKING: + # Declared, never defined: the host class this is mixed into owns + # every one of these. The block is stripped at runtime, so nothing + # here can shadow what the host actually binds. + _pc: Optional["RTCPeerConnection"] + _config: "WebRTCConfig" + _viewer_video_task: Optional[asyncio.Task] + _opus_audio_receiver: Optional["OpusMicReceiver"] + _send_ctrl: Callable[[Mapping[str, Any]], None] + _spawn_bg: Callable[[Any], asyncio.Task] + _consume_viewer_video: Callable[[Any], Coroutine[Any, Any, None]] + _start_opus_audio_receive: Callable[[Any], None] + def _maybe_resubscribe_viewer_video(self) -> None: if not (self._config.accept_viewer_video and self._viewer_video_task is None diff --git a/je_auto_control/utils/remote_desktop/webrtc_transport.py b/je_auto_control/utils/remote_desktop/webrtc_transport.py index b84d1722..d7f25a95 100644 --- a/je_auto_control/utils/remote_desktop/webrtc_transport.py +++ b/je_auto_control/utils/remote_desktop/webrtc_transport.py @@ -8,11 +8,12 @@ from __future__ import annotations import asyncio +import sys import threading import time from concurrent.futures import Future, ThreadPoolExecutor from dataclasses import dataclass, field -from typing import List, Optional, Sequence +from typing import Dict, List, Optional, Sequence, Tuple, TypedDict from je_auto_control.utils.logging.logging_instance import autocontrol_logger @@ -55,7 +56,14 @@ # clamp is the most reliable cross-codec lever we have without dropping # into encoder-specific options. "Auto" returns (24, "auto") and the # caller should treat it as "use defaults / pick from observed RTT". -BANDWIDTH_PRESETS = { +class BandwidthPreset(TypedDict): + """One entry of :data:`BANDWIDTH_PRESETS`: a frame rate and its label.""" + + fps: int + label: str + + +BANDWIDTH_PRESETS: Dict[str, BandwidthPreset] = { "auto": {"fps": 24, "label": "Auto"}, "low": {"fps": 10, "label": "Low (cellular)"}, "mid": {"fps": 18, "label": "Medium"}, @@ -112,30 +120,32 @@ def __init__(self) -> None: self._thread: Optional[threading.Thread] = None self._lock = threading.Lock() - def start(self) -> None: + def start(self) -> asyncio.AbstractEventLoop: + """Start the background loop if it isn't running; return it either way.""" with self._lock: - if self._loop is not None: - return - self._loop = asyncio.new_event_loop() + loop = self._loop + if loop is not None: + return loop + loop = asyncio.new_event_loop() + self._loop = loop self._thread = threading.Thread( - target=self._run, name="webrtc-loop", daemon=True, + target=self._run, args=(loop,), name="webrtc-loop", daemon=True, ) self._thread.start() autocontrol_logger.info("webrtc bridge: event loop started") + return loop - def _run(self) -> None: - asyncio.set_event_loop(self._loop) - self._loop.run_forever() + def _run(self, loop: asyncio.AbstractEventLoop) -> None: + asyncio.set_event_loop(loop) + loop.run_forever() def submit(self, coro) -> Future: """Schedule a coroutine; returns ``concurrent.futures.Future``.""" - self.start() - return asyncio.run_coroutine_threadsafe(coro, self._loop) + return asyncio.run_coroutine_threadsafe(coro, self.start()) def call_soon(self, callback, *args) -> None: """Schedule a sync callable from any thread.""" - self.start() - self._loop.call_soon_threadsafe(callback, *args) + self.start().call_soon_threadsafe(callback, *args) def stop(self) -> None: with self._lock: @@ -162,18 +172,17 @@ def get_bridge() -> _AsyncioBridge: _capture_local = threading.local() -def _get_cursor_position() -> Optional[tuple]: +def _get_cursor_position() -> Optional[Tuple[int, int]]: """Return absolute (x, y) cursor position, or None on unsupported platforms.""" - import sys as _sys try: - if _sys.platform == "win32": + if sys.platform == "win32": import ctypes from ctypes import wintypes point = wintypes.POINT() if ctypes.windll.user32.GetCursorPos(ctypes.byref(point)): return point.x, point.y return None - if _sys.platform == "darwin": + if sys.platform == "darwin": from Quartz import CGEventSourceGetMouseState # type: ignore location = CGEventSourceGetMouseState(0) return int(location.x), int(location.y) diff --git a/je_auto_control/utils/remote_desktop/webrtc_viewer.py b/je_auto_control/utils/remote_desktop/webrtc_viewer.py index 207e6674..72ee7331 100644 --- a/je_auto_control/utils/remote_desktop/webrtc_viewer.py +++ b/je_auto_control/utils/remote_desktop/webrtc_viewer.py @@ -9,7 +9,7 @@ import asyncio import json import threading -from typing import Any, Callable, Mapping, Optional +from typing import TYPE_CHECKING, Any, Callable, Mapping, Optional from je_auto_control.utils.logging.logging_instance import autocontrol_logger from je_auto_control.utils.remote_desktop.webrtc_transport import ( @@ -17,6 +17,19 @@ get_bridge, wait_for_ice_gathering, ) +if TYPE_CHECKING: # imported lazily at runtime to keep startup cheap + from je_auto_control.utils.remote_desktop.webrtc_audio import ( + OpusMicAudioTrack, OpusMicReceiver, + ) + from je_auto_control.utils.remote_desktop.webrtc_files import ( + FileTransferReceiver, + ) + from je_auto_control.utils.remote_desktop.webrtc_mic import MicUplinkSender + from je_auto_control.utils.remote_desktop.webrtc_transport import ( + ScreenVideoTrack, + ) + from je_auto_control.utils.usb.passthrough import UsbChannelClient + _OFFER_TIMEOUT_S = 12.0 @@ -27,6 +40,7 @@ FingerprintCallback = Callable[[str], None] InboxListingCallback = Callable[[list], None] InboxOpResultCallback = Callable[[str, bool, Optional[str]], None] +FileReceivedCallback = Callable[[Any], None] class WebRTCDesktopViewer: @@ -54,19 +68,19 @@ def __init__(self, *, token: str, self._on_auth_result = on_auth_result self._on_fingerprint = on_fingerprint self._pc: Optional[RTCPeerConnection] = None - self._control_channel = None - self._mic_channel = None - self._mic_sender = None # Optional[MicUplinkSender] - self._files_channel = None - self._files_receiver = None # Optional[FileTransferReceiver] - self._usb_channel = None - self._usb_client = None # Optional[UsbChannelClient] - self._on_file_received = None + self._control_channel: Any = None + self._mic_channel: Any = None + self._mic_sender: Optional[MicUplinkSender] = None + self._files_channel: Any = None + self._files_receiver: Optional[FileTransferReceiver] = None + self._usb_channel: Any = None + self._usb_client: Optional[UsbChannelClient] = None + self._on_file_received: Optional[FileReceivedCallback] = None self._on_inbox_listing: Optional[InboxListingCallback] = None self._on_inbox_op_result: Optional[InboxOpResultCallback] = None - self._viewer_screen_track = None - self._opus_audio_track = None - self._host_voice_receiver = None # OpusMicReceiver-like + self._viewer_screen_track: Optional[ScreenVideoTrack] = None + self._opus_audio_track: Optional[OpusMicAudioTrack] = None + self._host_voice_receiver: Optional[OpusMicReceiver] = None self._receive_task: Optional[asyncio.Task] = None self._authenticated = False self._read_only = False @@ -76,6 +90,13 @@ def __init__(self, *, token: str, # they finish (S7502). Tasks self-discard via a done callback. self._background_tasks: set = set() + def _require_pc(self) -> RTCPeerConnection: + """Return the live peer connection, or raise if there isn't one yet.""" + pc = self._pc + if pc is None: + raise RuntimeError("viewer has no peer connection; connect first") + return pc + def _spawn_bg(self, coro) -> "asyncio.Task": task = asyncio.ensure_future(coro) self._background_tasks.add(task) @@ -266,12 +287,14 @@ def _wire_files_channel(self, channel) -> None: from je_auto_control.utils.remote_desktop.webrtc_files import ( FileTransferReceiver, ) - if self._files_receiver is None: - self._files_receiver = FileTransferReceiver() + receiver = self._files_receiver + if receiver is None: + receiver = FileTransferReceiver() + self._files_receiver = receiver @channel.on("message") def _on_message(message) -> None: - self._files_receiver.handle_message( + receiver.handle_message( message, on_done=self._on_viewer_file_done, ) @@ -357,7 +380,7 @@ def _attach_viewer_screen_track(self) -> None: ScreenVideoTrack, ) video_transceivers = [ - t for t in self._pc.getTransceivers() if t.kind == "video" + t for t in self._require_pc().getTransceivers() if t.kind == "video" ] if len(video_transceivers) < 2: autocontrol_logger.warning( @@ -382,7 +405,7 @@ def _attach_opus_audio_track(self) -> None: OpusMicAudioTrack, ) audio_transceivers = [ - t for t in self._pc.getTransceivers() if t.kind == "audio" + t for t in self._require_pc().getTransceivers() if t.kind == "audio" ] if not audio_transceivers: autocontrol_logger.warning( @@ -505,11 +528,12 @@ def _start_host_voice_play(self, track) -> None: if self._host_voice_receiver is not None: return try: - self._host_voice_receiver = OpusMicReceiver() + receiver = OpusMicReceiver() except (RuntimeError, OSError) as error: autocontrol_logger.warning("host voice play init: %r", error) return - self._host_voice_receiver.consume(track) + self._host_voice_receiver = receiver + receiver.consume(track) autocontrol_logger.info("webrtc viewer: playing host voice") async def _consume_video(self, track) -> None: diff --git a/test/verify/typing_contract_exempt.txt b/test/verify/typing_contract_exempt.txt index 846f2cf4..e4a1a94b 100644 --- a/test/verify/typing_contract_exempt.txt +++ b/test/verify/typing_contract_exempt.txt @@ -14,7 +14,7 @@ # "Does not type-check" means on at least one of the three targets mypy can be # pointed at (win32, linux, darwin) — not just the one CI happens to run on. # -# Measured entries: 136 +# Measured entries: 123 je_auto_control.__main__ je_auto_control.cli @@ -100,19 +100,6 @@ je_auto_control.utils.rate_limit.rate_limit je_auto_control.utils.recording_edit.editor je_auto_control.utils.redaction.policies je_auto_control.utils.redaction.rules -je_auto_control.utils.remote_desktop.audio -je_auto_control.utils.remote_desktop.host_capture -je_auto_control.utils.remote_desktop.host_service -je_auto_control.utils.remote_desktop.jpeg_recorder -je_auto_control.utils.remote_desktop.session_recorder -je_auto_control.utils.remote_desktop.totp -je_auto_control.utils.remote_desktop.video_codec -je_auto_control.utils.remote_desktop.viewer -je_auto_control.utils.remote_desktop.webrtc_host -je_auto_control.utils.remote_desktop.webrtc_host_auth -je_auto_control.utils.remote_desktop.webrtc_host_media -je_auto_control.utils.remote_desktop.webrtc_transport -je_auto_control.utils.remote_desktop.webrtc_viewer je_auto_control.utils.rest_api.rest_server je_auto_control.utils.rotated_match.rotated_match je_auto_control.utils.run_history.history_store From 9cde59d6f6c765e24bc39b0fab24ad00a2d373d4 Mon Sep 17 00:00:00 2001 From: JeffreyChen Date: Sat, 22 Aug 2026 17:08:27 +0800 Subject: [PATCH 10/19] Clear fifteen more modules off the typing contract MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The accessibility backends, observability, the triggers, chatops, the REST server, the MCP HTTP transport and element_repository, together, because they kept hitting the same causes. Thirty-four of the forty-three accessibility errors were one missing return annotation: `_unsupported` always raises, so every method whose last statement is a call to it looked like it could fall through. It is `NoReturn` now. A catch tuple that is not typed as one catches nothing as far as mypy is concerned. `_uia_errors()` returned `Tuple[type, ...]`, which is not "a tuple of exception classes", so all five `except UIA_ERRORS` sites were errors — on a tuple that exists precisely because `comtypes.COMError` inherits from `Exception` and nothing else. `rest_server` and `chatops.router` had the same shape via `except (…, *SQLITE_ERRORS)`, which mypy cannot follow into an `except`; both now name the set once as an annotated module constant, as `mcp_server._protocol` already did. `parse_content_length` declared `Mapping[str, str]` and no caller ever passed one: all three hand it `BaseHTTPRequestHandler.headers`, an `email.message.Message`, which is not a mapping over its keys and which matches header names case-insensitively — the property that makes `Content-length` work. It takes a one-method protocol now. Four fixes are behaviour: * `Gauge` and `Histogram` borrowed `Counter._labels_key` by assignment, so a gauge's label typo was validated by a method declared to take a counter. The rule is the same for all three and moved to the base, which also gives the registry's `render()` loop a base to call. * IMAP UIDs were decoded at two of the three places that needed them. They are decoded once on arrival now, so the fetch, the mark-seen and the seen-set all speak one type. * `ElementRepository` passed a user-editable locator to the accessibility API as `**kwargs`, so a field that is not a filter came back as a TypeError about keyword arguments from inside the backend. * `_AtspiConnection._call` had no bus to call outside its `with`. `_process_name` is a kernel32 round trip and now says so with a `sys.platform` guard, instead of being type-checked against Linux. --- Progress.md | 61 +++++++++++++++---- WHATS_NEW.md | 58 ++++++++++++++++++ architecture_explore.md | 48 +++++++-------- .../utils/accessibility/backends/base.py | 4 +- .../accessibility/backends/linux_backend.py | 15 ++++- .../accessibility/backends/windows_backend.py | 9 ++- .../accessibility/backends/windows_query.py | 13 ++-- .../utils/accessibility/recorder.py | 8 ++- je_auto_control/utils/chatops/router.py | 19 +++--- .../element_repository/element_repository.py | 21 ++++++- je_auto_control/utils/http_headers.py | 17 +++++- .../utils/mcp_server/http_transport.py | 9 ++- .../utils/observability/exporter.py | 7 ++- .../utils/observability/metrics.py | 46 ++++++++------ .../utils/observability/tracing.py | 2 +- je_auto_control/utils/rest_api/rest_server.py | 30 ++++++--- .../utils/triggers/email_trigger.py | 36 ++++++----- .../utils/triggers/trigger_engine.py | 7 ++- .../utils/triggers/webhook_server.py | 7 +-- test/unit_test/headless/test_email_trigger.py | 13 ++-- .../headless/test_r3_net_triggers.py | 2 +- test/verify/typing_contract_exempt.txt | 17 +----- 22 files changed, 308 insertions(+), 141 deletions(-) diff --git a/Progress.md b/Progress.md index 886fa86a..63465ece 100644 --- a/Progress.md +++ b/Progress.md @@ -240,11 +240,11 @@ capability enum 值與 variadic `ei_seat_bind_capabilities`、event-type enum (`--cov-report=term-missing` 已經開著,CI 每一格都存了 `coverage.xml` artifact), 而不是齊頭式地補測試。 -### mypy:整包把關,123 個模組還沒過 +### mypy:整包把關,108 個模組還沒過 範圍不再是兩條路徑,而是**整包減去一張只准變少的清單** (`test/verify/typing_contract_exempt.txt`)。差別在於預設值:路徑清單只有人想到才會長, -新模組預設在圈外;現在新模組**預設就在契約裡**,1,017 個檔案有 894 個已經過關。 +新模組預設在圈外;現在新模組**預設就在契約裡**,1,017 個檔案有 909 個已經過關。 `wrapper` 那一群(6 個模組)在 2026-08-21 清掉了,做法見 [WHATS_NEW.md](WHATS_NEW.md): 平台縫的八個匯出名稱先宣告型別、再讓分支去綁定,其中三個用 @@ -286,15 +286,32 @@ Windows 上一直是 import 就炸;`win32_ctype_input` 則有一批標錯的 `base64.binascii.Error`(那個名字只是 `base64` 自己 import 的副作用)。 做法見 [WHATS_NEW.md](WHATS_NEW.md)。 -剩下 123 個模組要清。錯誤碼分布是 `attr-defined` 佔大宗,其次 +同日再清掉十五個:`utils/accessibility` 那一群(5)、`utils/observability`(3)、 +`utils/triggers`(3)、`utils/chatops/router`、`utils/rest_api/rest_server`、 +`utils/mcp_server/http_transport`、`utils/element_repository`。四十三個 +accessibility 錯誤裡有三十四個是同一個沒標的回傳型別(`_unsupported` 一定會拋, +標成 `NoReturn` 就全清);另一個反覆出現的是**攔截用的 tuple 沒有被標成 tuple** +(`Tuple[type, ...]` 不算「a tuple of exception classes」, +`except (…, *SQLITE_ERRORS)` 的星號解包 mypy 也跟不進 `except`)。 +順帶修掉四個行為問題:`Gauge`/`Histogram` 用 +`_labels_key = Counter._labels_key` 借方法、`parse_content_length` 宣告的 +`Mapping[str, str]` 三個呼叫端一個都沒給過(給的是 `email.message.Message`)、 +`ElementRepository` 把使用者可編輯的 locator 直接 `**kwargs` 丟進無障礙 API、 +`_AtspiConnection._call` 在 `with` 之外沒有 bus 可以呼叫。 +做法見 [WHATS_NEW.md](WHATS_NEW.md)。 + +剩下 108 個模組要清。錯誤碼分布是 `attr-defined` 佔大宗,其次 `arg-type`/`union-attr`/`assignment`,成群集中在 -`gui`(11+`gui/remote_desktop` 2)、 -`utils/accessibility/backends`/`utils/mcp_server`/`utils/usb/passthrough`(各 4)、 -`utils/observability`/`utils/triggers`(各 3),其餘散在各處。 +`gui`(11+`gui/remote_desktop` 2)、`utils/mcp_server`(4)、 +`utils/usb/passthrough`(4)、`utils/clipboard`(2),其餘散在各處。 +以檔案計最大的三個是 `utils/usb/passthrough/winusb_backend.py`(34)、 +`gui/_auto_click_tab.py`(31)、`gui/remote_desktop/webrtc_panel.py`(27)。 **下一步**是一次清一個群集,清完把行從清單刪掉—— `python test/verify/typing_contract_verify.py --fix` 會替你改,CI 會在你忘了刪的時候紅掉。 -剛清掉的那一群留下一個可以直接套用的樣板:mixin 用 `if TYPE_CHECKING:` 宣告 -借來的成員,`Optional` 屬性用 `TYPE_CHECKING` 匯入真正的型別去標注。 +已經清掉的那些留下四個可以直接套用的樣板:mixin 用 `if TYPE_CHECKING:` 宣告 +借來的成員、`Optional` 屬性用 `TYPE_CHECKING` 匯入真正的型別去標注、 +一定會拋的 helper 標 `NoReturn`、攔截用的 tuple 標 +`Tuple[Type[BaseException], ...]` 並收成一個模組常數(不要在 `except` 裡星號解包)。 #### 平台縫還缺的一半:`keyboard` 與 `mouse` 還沒有合約 @@ -323,12 +340,29 @@ Protocol 一旦標上去,它們的內部型別錯誤就會一起浮出來。`l (見上),只剩下面那條 `DECIDE` 的 ctypes 表面問題。所以前置條件實質上已經到位, 可以開始標 Protocol;下一次動這條的人不必再等別的群集。 -#### `DECIDE`:Windows 後端剩下的豁免不是它們的錯 +#### `DECIDE`:16 個模組的豁免不是它們的錯 + +**2026-08-22 重新實測,這一條的規模比原本寫的大一倍,而且形狀不同。** +原本寫的是「`windows/` 底下 8 個模組」;實際上是 **16 個模組、58 個錯誤**, +而其中**一半不在 `windows/` 底下**: + +| 模組 | 錯誤數 | +| --- | ---: | +| `windows/record/win32_input_hook.py` | 8 | +| `utils/usb/passthrough/key_provider.py` | 8 | +| `windows/core/utils/win32_ctype_input.py` | 6 | +| `windows/core/utils/win32_keypress_check.py`、`windows/interception/_dll.py`、`windows/mouse/win32_ctype_mouse_control.py`、`windows/screen/win32_screen.py`、`windows/window/windows_window_manage.py` | 各 4 | +| `windows/interception/mouse.py`、`gui/main_window.py`、`utils/app_idle/`、`utils/file_assoc/`、`utils/idle_keepawake/`、`utils/lock_session/`、`utils/session_guard/`、`utils/trash/` | 各 2 | + +這 16 個模組在 `--platform win32` 上**全部乾淨**,在 linux/darwin 上的錯誤 +**100% 是同一件事**:`ctypes` 的 Win32 專屬表面(`windll`、`WinDLL`、 +`WINFUNCTYPE`、`WinError`、`get_last_error`)在 typeshed 的非 Windows 目標上 +根本沒有宣告。實測方式是把三個平台的結果對比,找出「win32 全綠、其他平台 +只錯在這組名字上」的模組——結果是 16 個,沒有一個是「順便還有別的錯」。 -`windows/` 底下還有 8 個模組留在清單上,而它們在 `--platform win32` 上**全部乾淨**。 -留在清單上的唯一原因是這個閘門會用三個目標平台各檢查一次,而 `ctypes` 的 -Win32 專屬表面(`windll`、`WinDLL`、`WINFUNCTYPE`、`WinError`、`get_last_error`) -在 typeshed 的 linux/darwin 目標上根本沒有宣告。共 18 處,分布在 8 個檔。 +**這件事會改動下面第四條的可行性**:`utils/app_idle/`、`utils/trash/`、 +`gui/main_window.py` 這些不在任何平台目錄底下,所以「照目錄決定用哪個平台量」 +的規則對它們無效。要嘛規則改成看模組內容(難自動判定),要嘛這八個走另一條路。 三條路都實測過,維護者挑一條: @@ -340,6 +374,7 @@ Win32 專屬表面(`windll`、`WinDLL`、`WINFUNCTYPE`、`WinError`、`get_las 第四條是改閘門本身:讓 `typing_contract_verify.py` 只用模組自己的平台去量 平台專屬模組(`windows/` 只用 win32、`osx/` 只用 darwin、`linux_*` 只用 linux)。 +(**但見上面那張表**:有八個受影響的模組不在平台目錄底下,照目錄分不到它們。) 這是三者裡最貼近事實的一條——拿 Linux 去檢查一個只跑得動在 Windows 的模組本來就沒有意義, 而閘門的 docstring 自己寫的多目標理由是「要讀得到平台分支後面的程式碼」,不是這個。 但它會改掉「清單上的模組=在每個支援平台上都還沒過」這個既定語意,**所以要維護者拍板**。 diff --git a/WHATS_NEW.md b/WHATS_NEW.md index 9e076e34..fc033c4e 100644 --- a/WHATS_NEW.md +++ b/WHATS_NEW.md @@ -208,6 +208,64 @@ comparison as a `DECIDE`, because the cleanest of them changes what the gate means rather than what the code says. **136 modules left, 881 of 1,017 files inside the contract.** +### Fifteen More Modules, and Four Errors That Were Wrong Rather Than Untyped + +The accessibility backends, the observability trio, the three triggers, +`chatops.router`, `rest_api.rest_server`, `mcp_server.http_transport` and +`element_repository` came off the list together, because they kept running into +the same handful of causes. + +**Thirty-four of the forty-three accessibility errors were one missing +annotation.** `AccessibilityBackend._unsupported` raises for every action a +backend cannot perform, but it declared no return type — so mypy read the calls +as expressions that might fall through, and reported "missing return statement" +in all thirty-four methods that end with one. It is annotated `NoReturn` now, +which is what its body has always done. + +**A catch tuple that is not typed as one catches nothing, as far as mypy is +concerned.** `_uia_errors()` returns the exception classes a UIA call can raise +— including `comtypes`' `COMError`, which inherits from `Exception` and from +nothing else, so an `except (OSError, AttributeError)` never contained it. The +tuple was annotated `Tuple[type, ...]`, which is not "a tuple of exception +classes", so all five `except UIA_ERRORS` sites were errors. Same shape in +`rest_server` and `chatops.router`, where `except (…, *SQLITE_ERRORS)` unpacks +a tuple mypy cannot follow into an `except`; both now name the whole set as one +annotated module constant, the way `mcp_server._protocol` already did. + +**`parse_content_length` never took the type it declared.** Its parameter said +`Mapping[str, str]`; all three callers pass `self.headers` from a +`BaseHTTPRequestHandler`, which is an `email.message.Message` — not a mapping +over its keys, and case-insensitive about header names, which is the property +that makes `Content-length` work. It takes a one-method `HeaderLookup` protocol +now, which is what it actually uses and what the callers actually have. + +Four fixes are behaviour: + +- **`Gauge` and `Histogram` borrowed `Counter._labels_key` by assignment** + (`_labels_key = Counter._labels_key`), so a label typo in a gauge was + validated by a method whose `self` was declared to be a counter. The rule is + identical for all three, so it moved to `_MetricBase` — which also gives + `MetricRegistry.render()` a `render` to call on the base it iterates. +- **`_search_uids` decoded each IMAP UID at two later call sites and not at the + third.** UIDs are now decoded once where they arrive, so `_fetch_message`, + `_mark_seen` and `_seen_uids` all speak the same type. The stub in + `test_email_trigger.py` had pinned one exact `uid()` call shape + (`args[1]`); it now normalises arguments the way `imaplib._command` does — + skip `None`, ASCII-encode `str` — so it stands in for the library instead of + for one caller. +- **`ElementRepository` handed a stored locator straight to the accessibility + API as `**kwargs`.** A repository file is user-editable, so a field that is + not a filter surfaced as a `TypeError` about keyword arguments from inside + the backend. `_require` now rejects unknown fields by name, and the three + filters are passed explicitly. +- **`_AtspiConnection._call` had no bus to call outside its `with`.** It raises + `DBusError` naming the mistake rather than an `AttributeError` on `None`. + +`_process_name` in the Windows accessibility backend also stopped being checked +against Linux and macOS: it is a `kernel32` round trip, and now says so with a +`sys.platform` guard mypy can prune, in place of a bare `if process_id <= 0`. +**108 modules left, 909 of 1,017 files inside the contract.** + ### The Biggest Remaining Cluster: Thirteen Modules Under `utils/remote_desktop` `Progress.md` named this one as the next step and as the largest group left diff --git a/architecture_explore.md b/architecture_explore.md index df3d3edd..087b411c 100644 --- a/architecture_explore.md +++ b/architecture_explore.md @@ -20,7 +20,7 @@ iOS(WebDriverAgent)。核心能力是滑鼠/鍵盤控制、影像辨識、 | 指標 | 數值 | | --- | ---: | | Python 模組總數(含周邊子專案) | 1,031 | -| 程式碼總行數 | 140,596 | +| 程式碼總行數 | 140,680 | | `je_auto_control/utils/` 子套件數 | 310 | | `AC_*` 動作指令數(`known_commands()` 實測) | 773 | | 套件門面 `__all__` 公開名稱數 | 1,238 | @@ -158,7 +158,7 @@ socket server 有 8 MiB 讀取上限與 30 秒 handler timeout。 | `je_auto_control/api/__init__.py` | 22 | 版本化整合進入點。 | | `je_auto_control/api/core.py` | 19 | **穩定無頭 API 門面**:只暴露 `execute_action`、`execute_action_with_vars`、`generate_code`、`run_diagnostics`、`create_failure_bundle`、`failure_bundle_on_error`、`FailureBundleOptions`。mypy 型別契約以此為起點,現已擴到整包(見「設定基線」)。 | | `je_auto_control/utils/deprecation.py` | 35 | 公開 API 的一致性棄用警告。 | -| `je_auto_control/utils/http_headers.py` | 32 | 入站 HTTP 標頭的共用防禦式解析。 | +| `je_auto_control/utils/http_headers.py` | 45 | 入站 HTTP 標頭的共用防禦式解析。 | | `je_auto_control/utils/sqlite_support.py` | 56 | 選用標準函式庫 `sqlite3` 的取用點:`require_sqlite3()`/`sqlite3_available()`/`SQLITE_ERRORS`。十個以 SQLite 存放狀態的子系統都經由這裡,所以 FreeBSD 這種把 `sqlite3` 另外包成 `databases/py-sqlite3` 的 Python 仍然 import 得起門面。 | ### 5.2 wrapper 抽象層 @@ -320,7 +320,7 @@ socket server 有 8 MiB 讀取上限與 30 秒 handler timeout。 ### 5.4.3 排程、觸發與背景監看 -> 11 個套件、約 3,544 行。 +> 11 個套件、約 3,550 行。 | 模組 | 行數 | 職責 | | --- | ---: | --- | @@ -331,7 +331,7 @@ socket server 有 8 MiB 讀取上限與 30 秒 handler timeout。 | `utils/recurrence/` | 324 | RFC 5545 重複規則解析與發生時間展開 | | `utils/scheduler/` | 352 | 間隔式與 cron 式的 action JSON 排程器 | | `utils/session_guard/` | 62 | 驅動輸入前先偵測工作階段是否已鎖定/非互動 | -| `utils/triggers/` | 1,146 | 事件驅動觸發引擎:影像/視窗/像素/檔案/webhook/IMAP 郵件 | +| `utils/triggers/` | 1,152 | 事件驅動觸發引擎:影像/視窗/像素/檔案/webhook/IMAP 郵件 | | `utils/voice/` | 87 | 語音指令路由:把辨識到的語句對應到 `AC_*` action list | | `utils/watchdog/` | 173 | 背景彈窗/中斷看門狗,供無人值守自動化 | | `utils/watcher/` | 78 | 無頭輪詢原語:滑鼠位置、像素顏色、log tail | @@ -437,12 +437,12 @@ socket server 有 8 MiB 讀取上限與 30 秒 handler timeout。 ### 5.4.7 無障礙樹與原生控制項 -> 16 個套件、約 4,279 行。 +> 16 個套件、約 4,313 行。 | 模組 | 行數 | 職責 | | --- | ---: | --- | | `utils/a11y_audit/` | 355 | 以無障礙樹 + OCR 進行無障礙與 i18n 稽核 | -| `utils/accessibility/` | 2,818 | 跨平台無障礙樹定位與錄製;Windows UIA/macOS AX/null 三後端。支援限定視窗(換搜尋起點,不是過濾)、逐節點可中斷走訪、`IUIAutomation2` 連線逾時、名稱子字串比對與排序、`control_get_state` 一次讀完值/勾選/選取/數值(密碼欄位不回內容) | +| `utils/accessibility/` | 2,835 | 跨平台無障礙樹定位與錄製;Windows UIA/macOS AX/null 三後端。支援限定視窗(換搜尋起點,不是過濾)、逐節點可中斷走訪、`IUIAutomation2` 連線逾時、名稱子字串比對與排序、`control_get_state` 一次讀完值/勾選/選取/數值(密碼欄位不回內容) | | `utils/ax_events/` | 29 | 反應式 UIA 事件等待(focus-changed) | | `utils/ax_props/` | 44 | 讀取豐富 UIA 屬性(enabled/offscreen/help/status/快捷鍵) | | `utils/ax_text/` | 102 | 透過 UIA TextPattern 取得原生文字(讀取/尋找/選取/屬性) | @@ -450,7 +450,7 @@ socket server 有 8 MiB 讀取上限與 30 秒 handler timeout。 | `utils/contrast_map/` | 120 | 取樣實際顏色以評定畫面文字的可讀性(WCAG) | | `utils/control_patterns/` | 88 | 延伸 UIA 控制項模式動作(Expand/Select/Range/Scroll) | | `utils/cvd_simulate/` | 125 | 模擬色覺缺陷並標示在該狀況下會撞色的顏色 | -| `utils/element_repository/` | 105 | 原生 UI 元素的具名定位器倉庫(object repository) | +| `utils/element_repository/` | 122 | 原生 UI 元素的具名定位器倉庫(object repository) | | `utils/focus_order/` | 95 | 鍵盤焦點順序:預期 Tab 序列、WCAG 稽核與設定焦點 | | `utils/legacy_accessible/` | 45 | MSAA 橋接,處理 UIA 無法建模的舊控制項 | | `utils/selection_view/` | 57 | 容器選取狀態與檢視切換(Selection/MultipleView 模式) | @@ -490,7 +490,7 @@ socket server 有 8 MiB 讀取上限與 30 秒 handler timeout。 ### 5.4.9 AI / Agent / LLM -> 13 個套件、約 20,610 行。 +> 13 個套件、約 20,617 行。 | 模組 | 行數 | 職責 | | --- | ---: | --- | @@ -503,7 +503,7 @@ socket server 有 8 MiB 讀取上限與 30 秒 handler timeout。 | `utils/cua_action/` | 127 | 標準化 computer-use 動作結構(Anthropic/OpenAI → `AC_*`) | | `utils/llm/` | 357 | 自然語言 → action list 規劃器 + Anthropic/null 後端 | | `utils/mcp_registry/` | 92 | MCP registry `server.json` 資訊清單產生(可被發現) | -| `utils/mcp_server/` | 17,323 | **無頭 MCP 伺服器**(16K LOC,預設註冊 676 個工具=657 個 `ac_*` + 19 個別名):stdio + HTTP 傳輸、工具工廠與處理器、資源、prompt、稽核、限流、外掛熱重載 | +| `utils/mcp_server/` | 17,330 | **無頭 MCP 伺服器**(16K LOC,預設註冊 676 個工具=657 個 `ac_*` + 19 個別名):stdio + HTTP 傳輸、工具工廠與處理器、資源、prompt、稽核、限流、外掛熱重載 | | `utils/tool_use_schema/` | 180 | 把 `AC_*` 指令匯出成 Claude/OpenAI 的 tool-use schema | | `utils/trajectory_eval/` | 106 | agent 軌跡評估:依評分規準為一次執行打分 | | `utils/vision/` | 448 | VLM 元素定位器(依描述找元素)+ Anthropic/OpenAI/null 後端 | @@ -523,12 +523,12 @@ socket server 有 8 MiB 讀取上限與 30 秒 handler timeout。 ### 5.4.11 伺服器、網路協定與外部整合 -> 24 個套件、約 5,900 行。 +> 24 個套件、約 5,917 行。 | 模組 | 行數 | 職責 | | --- | ---: | --- | | `utils/acme_v2/` | 598 | 完整 ACME v2 用戶端(RFC 8555),不依賴 certbot | -| `utils/chatops/` | 628 | Chat-ops bot:接收 Slack/Discord/webhook 的 slash 指令並路由到動作 | +| `utils/chatops/` | 633 | Chat-ops bot:接收 Slack/Discord/webhook 的 slash 指令並路由到動作 | | `utils/cookie_jar/` | 103 | RFC 6265 cookie jar | | `utils/email_send/` | 116 | SMTP 寄信(email 觸發器的發送端搭檔) | | `utils/events/` | 82 | 對外 CloudEvents 發送(執行生命週期事件) | @@ -545,7 +545,7 @@ socket server 有 8 MiB 讀取上限與 30 秒 handler timeout。 | `utils/otp/` | 37 | TOTP 一次性密碼產生(自動化 2FA 登入) | | `utils/outbox/` | 92 | 交易式 outbox,保證至少一次的事件投遞 | | `utils/pytest_plugin/` | 373 | pytest 外掛 + BDD step library(`pytest11` entry point) | -| `utils/rest_api/` | 1,739 | 純標準庫 REST 前端:路由、Bearer 驗證、限流、Prometheus 指標、OpenAPI 3.1 產生 | +| `utils/rest_api/` | 1,751 | 純標準庫 REST 前端:路由、Bearer 驗證、限流、Prometheus 指標、OpenAPI 3.1 產生 | | `utils/socket_server/` | 131 | 執行 action JSON 的執行緒式 TCP 指令伺服器(預設綁 127.0.0.1) | | `utils/sse_client/` | 112 | Server-Sent Events 用戶端解析 | | `utils/tls_acme/` | 447 | TLS 自動化:HTTP-01 挑戰伺服器、金鑰/CSR、自動續期 | @@ -554,7 +554,7 @@ socket server 有 8 MiB 讀取上限與 30 秒 handler timeout。 ### 5.4.12 報表、可觀測性與測試治理 -> 34 個套件、約 6,896 行。 +> 34 個套件、約 6,903 行。 | 模組 | 行數 | 職責 | | --- | ---: | --- | @@ -571,7 +571,7 @@ socket server 有 8 MiB 讀取上限與 30 秒 handler timeout。 | `utils/flakiness/` | 150 | 以執行歷史分析不穩定測試 | | `utils/generate_report/` | 310 | HTML/JSON/XML 三種報表產生器(Template Method) | | `utils/media_assert/` | 233 | 媒體斷言:音訊活動與影片動態檢查 | -| `utils/observability/` | 661 | Prometheus 格式指標 + OpenTelemetry 相容 trace + `/metrics` 匯出伺服器 | +| `utils/observability/` | 668 | Prometheus 格式指標 + OpenTelemetry 相容 trace + `/metrics` 匯出伺服器 | | `utils/otlp_export/` | 81 | OTLP/JSON span 匯出 | | `utils/percentiles/` | 103 | 可合併的串流延遲摘要與精確百分位數 | | `utils/process_doc/` | 85 | 由錄製的 action list 產生逐步 SOP 文件 | @@ -702,14 +702,14 @@ socket server 有 8 MiB 讀取上限與 30 秒 handler timeout。 | `action_schema.py` | 128 | action list 的結構驗證:形狀、參數型別、未知指令拒絕。單一走訪同時支援兩種消費方式:`validate_actions()` 遇到第一個問題就拋、`unknown_command_names()` 收齊全部不認得的名字(REST `/execute` 用它回 400)。 | | `mouse_aliases.py` | 39 | 單鍵點擊別名(`AC_click_left` 等),executor 與 callback executor 共用。 | -#### `utils/mcp_server/`(17,323 行,676 個工具)— 最大子系統 +#### `utils/mcp_server/`(17,330 行,676 個工具)— 最大子系統 | 檔案 | 行數 | 職責 | | --- | ---: | --- | | `tools/_factories.py` | 8,739 | 工具工廠:每個函式回傳一個領域的 `MCPTool` 清單(把 `AC_*` 能力包成 MCP 工具)。 | | `tools/_handlers.py` | 4,651 | 把 MCP 工具呼叫橋接到 AutoControl 無頭 API 的 adapter。 | | `server.py` | 713 | JSON-RPC 2.0 over stdio 的最小 MCP 伺服器:連線範圍狀態、行內/併發分派、工具與 resource/prompt 處理器。 | -| `http_transport.py` | 514 | MCP 的 HTTP 傳輸。 | +| `http_transport.py` | 521 | MCP 的 HTTP 傳輸。 | | `http_sessions.py` | 234 | MCP 的 HTTP 傳輸用的 session 身分:`Mcp-Session-Id` 註冊表,以及每個 session 那條常駐的 server→client SSE 串流。 | | `_client_requests.py` | 217 | 伺服器主動送出的請求:`roots/list`/`elicitation/create`/`sampling/createMessage`,對應表與回應路由,以及破壞性工具的確認交握。 | | `_protocol.py` | 165 | JSON-RPC 線路格式:版本與識別常數、`_MCPError`、決定失敗工具行為的錯誤 tuple、envelope 產生器、工具回傳值轉 `content` 區塊。不碰伺服器狀態。 | @@ -800,11 +800,11 @@ socket server 有 8 MiB 讀取上限與 30 秒 handler timeout。 | `usbip/libusb_backend.py` | 209 | 以 PyUSB/libusb 執行 URB 的正式後端。 | | `usbip/backend.py` | 88 | 可插拔 URB 執行後端。 | -#### `utils/rest_api/`(1,739 行) +#### `utils/rest_api/`(1,751 行) | 檔案 | 行數 | 職責 | | --- | ---: | --- | -| `rest_server.py` | 468 | HTTP 前端主體。 | +| `rest_server.py` | 480 | HTTP 前端主體。 | | `rest_handlers.py` | 486 | 端點實作。 | | `rest_openapi.py` | 422 | 走訪路由表產生 OpenAPI 3.1 規格。 | | `rest_auth.py` | 143 | Bearer token 驗證 + 逐 client 限流閘門。 | @@ -1025,25 +1025,25 @@ socket 預設綁 `127.0.0.1`;資源一律用 `with`。 | 層/子系統 | 檔案數 | 行數 | | --- | ---: | ---: | | `gui/` | 89 | 26,542 | -| `utils/mcp_server/` | 21 | 17,323 | +| `utils/mcp_server/` | 21 | 17,330 | | `utils/remote_desktop/` | 56 | 11,990 | | `utils/executor/` | 6 | 9,075 | | `utils/usb/` | 17 | 4,250 | | `je_auto_control/`(頂層 3 檔) | 3 | 2,363 | -| `utils/accessibility/` | 13 | 2,818 | +| `utils/accessibility/` | 13 | 2,835 | | `wrapper/` | 19 | 3,293 | | `windows/` | 23 | 1,901 | -| `utils/rest_api/` | 8 | 1,739 | +| `utils/rest_api/` | 8 | 1,751 | | `utils/agent/` | 8 | 1,250 | | `linux_with_x11/` | 19 | 1,236 | | `linux_wayland/` | 17 | 2,870 | -| `utils/triggers/` | 4 | 1,146 | +| `utils/triggers/` | 4 | 1,152 | | `utils/ocr/` | 9 | 1,112 | | `utils/usbip/` | 5 | 920 | | `utils/assertion/` | 3 | 863 | | `osx/` | 17 | 915 | | `autocontrol-lsp/` | 8 | 744 | | `utils/hotkey/` | 7 | 727 | -| 其餘模組(約 286 個 `utils/` 子套件 + `android/`/`ios/`/周邊小工具) | 673 | 47,454 | -| **總計** | **1,025** | **140,531** | +| 其餘模組(約 286 個 `utils/` 子套件 + `android/`/`ios/`/周邊小工具) | 673 | 47,496 | +| **總計** | **1,025** | **140,615** | diff --git a/je_auto_control/utils/accessibility/backends/base.py b/je_auto_control/utils/accessibility/backends/base.py index 4fe19e19..a7685100 100644 --- a/je_auto_control/utils/accessibility/backends/base.py +++ b/je_auto_control/utils/accessibility/backends/base.py @@ -1,5 +1,5 @@ """Abstract accessibility backend.""" -from typing import Any, Dict, List, Optional +from typing import Any, Dict, List, NoReturn, Optional from je_auto_control.utils.accessibility.element import ( AccessibilityElement, AccessibilityNotAvailableError, @@ -323,7 +323,7 @@ def wait_for_focus_change(self, timeout: float = 5.0, """ self._unsupported("wait_for_focus_change", timeout) - def _unsupported(self, operation: str, *context: Any): + def _unsupported(self, operation: str, *context: Any) -> NoReturn: """Raise a clear error for an action this backend can't perform.""" raise AccessibilityNotAvailableError( f"{operation} is not supported by the {self.name} backend", diff --git a/je_auto_control/utils/accessibility/backends/linux_backend.py b/je_auto_control/utils/accessibility/backends/linux_backend.py index 58cb1412..0c986a1f 100644 --- a/je_auto_control/utils/accessibility/backends/linux_backend.py +++ b/je_auto_control/utils/accessibility/backends/linux_backend.py @@ -113,12 +113,23 @@ def _address() -> str: def root(self) -> Reference: return (_REGISTRY, _ROOT_PATH) + def _require_bus(self) -> SessionBus: + """Return the live bus, or say which mistake was made.""" + bus = self._bus + if bus is None: + raise DBusError( + "AT-SPI connection is not open; use it as a context manager", + ) + return bus + def _call(self, reference: Reference, interface: str, member: str, signature: str = "", body: Optional[List[Any]] = None, timeout: float = 10.0) -> List[Any]: sender, path = reference - return self._bus.call(sender, path, interface, member, - signature, body or [], timeout=timeout) + return self._require_bus().call( + sender, path, interface, member, + signature, body or [], timeout=timeout, + ) # --- reads ------------------------------------------------------------- diff --git a/je_auto_control/utils/accessibility/backends/windows_backend.py b/je_auto_control/utils/accessibility/backends/windows_backend.py index a264b1b9..0d2783f4 100644 --- a/je_auto_control/utils/accessibility/backends/windows_backend.py +++ b/je_auto_control/utils/accessibility/backends/windows_backend.py @@ -9,6 +9,7 @@ decorative text children. """ import functools +import sys from typing import Any, Dict, List, Optional from je_auto_control.utils.accessibility.backends.base import ( @@ -113,8 +114,10 @@ class WindowsAccessibilityBackend(AccessibilityBackend): def __init__(self) -> None: import threading self.available = _is_available() - self._automation = None - self._uia_module = None + self._automation: Any = None + # The comtypes-generated UIAutomationClient module; `Any` because it + # is generated at import time and has no declarations to check. + self._uia_module: Any = None self._event_lock = threading.Lock() def _ensure_automation(self): @@ -892,7 +895,7 @@ def _process_name(process_id: int) -> str: recycle pids, so a very long-lived session could in principle read a stale name here; it only labels ``app_name``, and the cache is bounded. """ - if process_id <= 0: + if process_id <= 0 or sys.platform != "win32": return "" try: import ctypes diff --git a/je_auto_control/utils/accessibility/backends/windows_query.py b/je_auto_control/utils/accessibility/backends/windows_query.py index adedb02a..08c6b1b1 100644 --- a/je_auto_control/utils/accessibility/backends/windows_query.py +++ b/je_auto_control/utils/accessibility/backends/windows_query.py @@ -17,7 +17,8 @@ Imports no ``PySide6``. """ -from typing import Any, Iterator, Optional, Tuple +import sys +from typing import Any, Iterator, List, Optional, Tuple, Type from je_auto_control.utils.accessibility.element import ( AccessibilityNotAvailableError, @@ -26,7 +27,7 @@ TREE_SCOPE_DESCENDANTS = 4 -def _uia_errors() -> Tuple[type, ...]: +def _uia_errors() -> Tuple[Type[BaseException], ...]: """Exception types a UIA call can raise. ``comtypes`` reports provider failures as ``COMError``, which inherits from @@ -35,12 +36,10 @@ def _uia_errors() -> Tuple[type, ...]: A window that closes mid-walk, or an application that stops responding, surfaces exactly that way. """ - errors: list = [OSError, AttributeError, ValueError] - try: + errors: List[Type[BaseException]] = [OSError, AttributeError, ValueError] + if sys.platform == "win32": from _ctypes import COMError - except ImportError: # non-Windows - return tuple(errors) - errors.append(COMError) + errors.append(COMError) return tuple(errors) diff --git a/je_auto_control/utils/accessibility/recorder.py b/je_auto_control/utils/accessibility/recorder.py index 11527922..8d140c5c 100644 --- a/je_auto_control/utils/accessibility/recorder.py +++ b/je_auto_control/utils/accessibility/recorder.py @@ -164,13 +164,17 @@ def _record(self, event: AXRecorderEvent) -> None: def _make_event(kind: str, snapshot: Dict[str, Any], kind_override: bool = False) -> AXRecorderEvent: - bounds = snapshot.get("bounds") or (0, 0, 0, 0) + # AXRecorderEvent.bounds is a four-tuple; a short or missing box pads + # with zeros rather than silently producing a tuple of another length. + box = [int(value) for value in list(snapshot.get("bounds") or ())[:4]] + box += [0] * (4 - len(box)) + left, top, width, height = box return AXRecorderEvent( timestamp_iso=_now_iso(), kind=kind, role=str(snapshot.get("role") or ""), name=str(snapshot.get("name") or ""), - bounds=tuple(int(v) for v in bounds[:4]), + bounds=(left, top, width, height), app_name=str(snapshot.get("app_name") or ""), details={"raw": dict(snapshot)} if kind_override else {}, ) diff --git a/je_auto_control/utils/chatops/router.py b/je_auto_control/utils/chatops/router.py index ede88c72..629504cb 100644 --- a/je_auto_control/utils/chatops/router.py +++ b/je_auto_control/utils/chatops/router.py @@ -12,12 +12,22 @@ import shlex import threading from dataclasses import asdict, dataclass, field -from typing import Any, Callable, Dict, List, Optional +from typing import Any, Callable, Dict, List, Optional, Tuple, Type from je_auto_control.utils.exception.exceptions import AutoControlException from je_auto_control.utils.sqlite_support import SQLITE_ERRORS +# What a command handler is allowed to fail with. The router is the +# containment boundary for those failures: a bad script +# (AutoControlException) or a run-history read error (a sqlite3 error) must +# come back as a chat reply, not escape and kill the transport's poll loop. +_HANDLER_ERRORS: Tuple[Type[BaseException], ...] = ( + RuntimeError, OSError, ValueError, TypeError, LookupError, + AttributeError, AutoControlException, *SQLITE_ERRORS, +) + + # Built-in commands always available even when the operator only # registers their own scripts. Keep this set small — listing scripts # and reporting status is universal enough to be the bot's "?" reply. @@ -136,16 +146,11 @@ def _dispatch_argv(self, argv: List[str], f"{spec.required_role!r}; you do not have it."), succeeded=False, ) - # The router is the containment boundary for handler failures: a bad - # script (AutoControlException) or a run-history read error - # (a sqlite3 error) must come back as a chat reply, not escape and kill - # the transport's poll loop. try: return spec.handler(rest, context) except ChatOpsError as error: return CommandResult(text=f"{name}: {error}", succeeded=False) - except (RuntimeError, OSError, ValueError, TypeError, LookupError, - AttributeError, AutoControlException, *SQLITE_ERRORS) as error: + except _HANDLER_ERRORS as error: return CommandResult( text=f"{name} failed: {type(error).__name__}: {error}", succeeded=False, diff --git a/je_auto_control/utils/element_repository/element_repository.py b/je_auto_control/utils/element_repository/element_repository.py index 6826e7e7..e0bd8da8 100644 --- a/je_auto_control/utils/element_repository/element_repository.py +++ b/je_auto_control/utils/element_repository/element_repository.py @@ -73,16 +73,30 @@ def all(self) -> Dict[str, Dict[str, str]]: return {key: dict(value) for key, value in self._items.items()} def _require(self, key: str) -> Dict[str, str]: + """Return the stored locator for ``key``, rejecting unknown fields. + + A repository file is user-editable, so a field no accessibility + filter accepts used to surface as a ``TypeError`` about keyword + arguments from inside the backend call. + """ locator = self.get(key) if locator is None: raise KeyError(f"no locator named {key!r}") + unknown = sorted(set(locator) - set(_FILTER_FIELDS)) + if unknown: + raise ValueError( + f"locator {key!r} has fields that are not accessibility " + f"filters: {', '.join(unknown)}") return locator def resolve(self, key: str) -> Any: """Find the live element for ``key`` (or ``None`` if not present).""" from je_auto_control.utils.accessibility import ( find_accessibility_element) - return find_accessibility_element(**self._require(key)) + locator = self._require(key) + return find_accessibility_element( + name=locator.get("name"), role=locator.get("role"), + app_name=locator.get("app_name")) def find_info(self, key: str) -> Dict[str, Any]: """Resolve ``key`` and return a serialisable summary.""" @@ -96,4 +110,7 @@ def click(self, key: str) -> bool: """Click the element for ``key``; return whether it matched.""" from je_auto_control.utils.accessibility import ( click_accessibility_element) - return click_accessibility_element(**self._require(key)) + locator = self._require(key) + return click_accessibility_element( + name=locator.get("name"), role=locator.get("role"), + app_name=locator.get("app_name")) diff --git a/je_auto_control/utils/http_headers.py b/je_auto_control/utils/http_headers.py index 62555948..d27a991e 100644 --- a/je_auto_control/utils/http_headers.py +++ b/je_auto_control/utils/http_headers.py @@ -6,7 +6,7 @@ died and the connection was closed with no response at all — the client saw a reset instead of the 400 each server already had code to send. """ -from typing import Mapping +from typing import Any, Protocol # Sentinel for "the client did not give us a usable length". It is negative on # purpose: every caller already rejects non-positive lengths, so an @@ -15,7 +15,20 @@ INVALID_CONTENT_LENGTH = -1 -def parse_content_length(headers: Mapping[str, str]) -> int: +class HeaderLookup(Protocol): + """Anything that answers ``get(name)`` for one HTTP header. + + Deliberately not ``Mapping[str, str]``: ``http.server`` hands each + handler an ``email.message.Message``, which is not a mapping over its + keys and which matches header names case-insensitively — the property + that makes ``Content-length`` work. Every caller here passes that. + """ + + def get(self, name: str, /) -> Any: + """Return the header's value, or ``None`` when it is absent.""" + + +def parse_content_length(headers: HeaderLookup) -> int: """Return the request's Content-Length, or ``INVALID_CONTENT_LENGTH``. Never raises: a malformed, negative, or absent header yields the sentinel. diff --git a/je_auto_control/utils/mcp_server/http_transport.py b/je_auto_control/utils/mcp_server/http_transport.py index d2969825..8bbe4597 100644 --- a/je_auto_control/utils/mcp_server/http_transport.py +++ b/je_auto_control/utils/mcp_server/http_transport.py @@ -473,7 +473,14 @@ def start(self) -> None: self._server.socket, server_side=True, do_handshake_on_connect=False, ) - self._address = self._server.server_address[:2] + # `server_address` is typed for every address family a socketserver + # can bind; an AF_INET HTTP server always answers with (host, port). + bound_host, bound_port = self._server.server_address[:2] + self._address = ( + bound_host.decode() if isinstance(bound_host, (bytes, bytearray)) + else bound_host, + int(bound_port), + ) self._thread = threading.Thread( target=self._server.serve_forever, daemon=True, name="AutoControlMCPHttp", diff --git a/je_auto_control/utils/observability/exporter.py b/je_auto_control/utils/observability/exporter.py index c0a4eb92..8ee17239 100644 --- a/je_auto_control/utils/observability/exporter.py +++ b/je_auto_control/utils/observability/exporter.py @@ -82,14 +82,15 @@ def start(self) -> int: return self._port def stop(self) -> None: - if not self.is_running: + server = self._server + if server is None: return try: - self._server.shutdown() + server.shutdown() except (OSError, RuntimeError): pass try: - self._server.server_close() + server.server_close() except OSError: pass if self._thread is not None: diff --git a/je_auto_control/utils/observability/metrics.py b/je_auto_control/utils/observability/metrics.py index 9c4fbffc..14478353 100644 --- a/je_auto_control/utils/observability/metrics.py +++ b/je_auto_control/utils/observability/metrics.py @@ -61,6 +61,32 @@ class _MetricBase: help_text: str label_names: Tuple[str, ...] = field(default_factory=tuple) + def _labels_key(self, labels: Optional[Dict[str, str]] + ) -> Tuple[Tuple[str, str], ...]: + """Validate a label set against ``label_names`` and freeze it. + + Every metric type validates labels the same way — a typo must not + quietly fork into a new series — so the rule lives here rather than + being copied off ``Counter`` by each subclass. + """ + if not self.label_names: + return () + if not labels: + raise ValueError( + f"{self.name} expects labels {self.label_names}", + ) + # Reject any label name not declared at registration. + unknown = set(labels) - set(self.label_names) + if unknown: + raise ValueError( + f"unknown labels {sorted(unknown)} for {self.name}", + ) + return _frozen_labels(labels) + + def render(self) -> str: + """Render this metric in the Prometheus text exposition format.""" + raise NotImplementedError + class Counter(_MetricBase): """Monotonically increasing counter. @@ -92,22 +118,6 @@ def value(self, *, labels: Optional[Dict[str, str]] = None) -> float: with self._lock: return self._values.get(key, 0.0) - def _labels_key(self, labels: Optional[Dict[str, str]] - ) -> Tuple[Tuple[str, str], ...]: - if not self.label_names: - return () - if not labels: - raise ValueError( - f"{self.name} expects labels {self.label_names}", - ) - # Reject any label name not declared at registration. - unknown = set(labels) - set(self.label_names) - if unknown: - raise ValueError( - f"unknown labels {sorted(unknown)} for {self.name}", - ) - return _frozen_labels(labels) - def render(self) -> str: lines: List[str] = [ f"# HELP {self.name} {self.help_text}", @@ -154,8 +164,6 @@ def value(self, *, labels: Optional[Dict[str, str]] = None) -> float: with self._lock: return self._values.get(key, 0.0) - _labels_key = Counter._labels_key # same validation rules - def render(self) -> str: lines: List[str] = [ f"# HELP {self.name} {self.help_text}", @@ -215,8 +223,6 @@ def observe(self, value: float, if value <= boundary: series.bucket_counts[idx] += 1 - _labels_key = Counter._labels_key - def snapshot(self, *, labels: Optional[Dict[str, str]] = None ) -> Dict[str, object]: key = self._labels_key(labels) diff --git a/je_auto_control/utils/observability/tracing.py b/je_auto_control/utils/observability/tracing.py index e9a40833..523b4d1b 100644 --- a/je_auto_control/utils/observability/tracing.py +++ b/je_auto_control/utils/observability/tracing.py @@ -147,7 +147,7 @@ def traced(span_name: Optional[str] = None, """Decorator: wrap a callable in a span. ``span_name`` defaults to ``f.__qualname__``.""" def decorator(fn: Callable[..., Any]) -> Callable[..., Any]: - name = span_name or getattr(fn, "__qualname__", fn.__name__) + name = str(span_name or getattr(fn, "__qualname__", fn.__name__)) @wraps(fn) def wrapper(*args, **kwargs): diff --git a/je_auto_control/utils/rest_api/rest_server.py b/je_auto_control/utils/rest_api/rest_server.py index 1582374e..3b7608fe 100644 --- a/je_auto_control/utils/rest_api/rest_server.py +++ b/je_auto_control/utils/rest_api/rest_server.py @@ -14,7 +14,7 @@ import threading from http.server import BaseHTTPRequestHandler, ThreadingHTTPServer from pathlib import Path -from typing import Any, Callable, Dict, Optional, Tuple +from typing import Any, Callable, Dict, Optional, Tuple, Type from urllib.parse import urlparse from je_auto_control.utils.exception.exceptions import AutoControlException @@ -40,6 +40,17 @@ from je_auto_control.utils.sqlite_support import SQLITE_ERRORS +# What a route handler is allowed to fail with. AutoControlException is the +# family base (every framework error derives from it). The sqlite3 errors are +# separate: handlers such as /history read the shared run-history DB, and a +# locked or corrupt DB otherwise escaped the handler thread and dropped the +# connection with no response. That tuple is empty on a Python built without +# sqlite3, which catches exactly the right amount there: nothing. +_HANDLER_ERRORS: Tuple[Type[BaseException], ...] = ( + OSError, RuntimeError, ValueError, TypeError, AutoControlException, + *SQLITE_ERRORS, +) + HandlerFn = Callable[[RouteContext], HandlerResult] _GET_ROUTES: Dict[str, HandlerFn] = { @@ -190,13 +201,7 @@ def _dispatch(self, method: str, routes: Dict[str, HandlerFn], ctx = RouteContext(query=parsed.query, body=body, client_ip=client_ip) try: status, payload = handler(ctx) - # AutoControlException is the family base (every framework error derives - # from it). The sqlite3 errors are separate: handlers such as /history - # read the shared run-history DB, and a locked/corrupt DB otherwise - # escaped the handler thread and dropped the connection with no - # response. The tuple is empty on a Python built without sqlite3. - except (OSError, RuntimeError, ValueError, TypeError, - AutoControlException, *SQLITE_ERRORS) as error: + except _HANDLER_ERRORS as error: autocontrol_logger.error( "rest-api %s %s handler raised: %r", method, parsed.path, error, ) @@ -386,7 +391,14 @@ def start(self) -> None: server.auth_gate = self._auth # type: ignore[attr-defined] server.audit_log = self._audit_log # type: ignore[attr-defined] server.metrics = self._metrics # type: ignore[attr-defined] - self._address = server.server_address[:2] + # `server_address` is typed for every address family a socketserver + # can bind; an AF_INET HTTP server always answers with (host, port). + bound_host, bound_port = server.server_address[:2] + self._address = ( + bound_host.decode() if isinstance(bound_host, (bytes, bytearray)) + else bound_host, + int(bound_port), + ) self._server = server self._thread = threading.Thread( target=server.serve_forever, daemon=True, name="AutoControlREST", diff --git a/je_auto_control/utils/triggers/email_trigger.py b/je_auto_control/utils/triggers/email_trigger.py index 5b1c2558..e33184f6 100644 --- a/je_auto_control/utils/triggers/email_trigger.py +++ b/je_auto_control/utils/triggers/email_trigger.py @@ -107,6 +107,7 @@ def _connect(trigger: EmailTrigger) -> imaplib.IMAP4: # Pin a modern TLS floor; create_default_context already does this on # 3.10+, but stating it explicitly satisfies python:S4423. context.minimum_version = ssl_module.TLSVersion.TLSv1_2 + client: imaplib.IMAP4 if trigger.use_ssl: client = imaplib.IMAP4_SSL(trigger.host, trigger.port, ssl_context=context) @@ -116,14 +117,20 @@ def _connect(trigger: EmailTrigger) -> imaplib.IMAP4: return client -def _search_uids(client: imaplib.IMAP4, criteria: str) -> List[bytes]: - typ, data = client.uid("SEARCH", None, criteria or "UNSEEN") +def _search_uids(client: imaplib.IMAP4, criteria: str) -> List[str]: + """Return the matching message UIDs. + + ``imaplib`` accepts the UID as either ``str`` or ``bytes`` and encodes + ASCII either way; the searched-for UIDs arrive as bytes and are decoded + here so every helper below takes one type. + """ + typ, data = client.uid("SEARCH", criteria or "UNSEEN") if typ != "OK" or not data or not data[0]: return [] - return data[0].split() + return [chunk.decode("ascii", "replace") for chunk in data[0].split()] -def _fetch_message(client: imaplib.IMAP4, uid: bytes): +def _fetch_message(client: imaplib.IMAP4, uid: str): typ, data = client.uid("FETCH", uid, "(RFC822)") if typ != "OK" or not data or data[0] is None: return None @@ -133,7 +140,7 @@ def _fetch_message(client: imaplib.IMAP4, uid: bytes): return email.message_from_bytes(bytes(raw), policy=email.policy.default) -def _mark_seen(client: imaplib.IMAP4, uid: bytes) -> None: +def _mark_seen(client: imaplib.IMAP4, uid: str) -> None: try: client.uid("STORE", uid, "+FLAGS", "(\\Seen)") except imaplib.IMAP4.error as error: @@ -151,10 +158,9 @@ def __init__(self, self._triggers: Dict[str, EmailTrigger] = {} self._stop = threading.Event() self._thread: Optional[threading.Thread] = None - if executor is None: - self._executor = self._default_executor - else: - self._executor = executor + self._executor: Callable[[list, Dict[str, Any]], Any] = ( + self._default_executor if executor is None else executor + ) @property def is_running(self) -> bool: @@ -296,10 +302,9 @@ def _poll_one(self, trigger: EmailTrigger) -> int: return fired def _iter_unprocessed_uids(self, client: imaplib.IMAP4, - trigger: EmailTrigger) -> Iterable[bytes]: + trigger: EmailTrigger) -> Iterable[str]: for uid in _search_uids(client, trigger.search_criteria): - uid_str = uid.decode("ascii", errors="replace") - if uid_str in trigger._seen_uids: + if uid in trigger._seen_uids: continue yield uid @@ -310,12 +315,11 @@ def _record_connect_error(self, trigger: EmailTrigger, trigger.trigger_id, error) def _fire_for_uid(self, client: imaplib.IMAP4, - trigger: EmailTrigger, uid: bytes) -> int: + trigger: EmailTrigger, uid: str) -> int: msg = _fetch_message(client, uid) if msg is None: return 0 - uid_str = uid.decode("ascii", errors="replace") - payload = _build_payload(uid_str, msg) + payload = _build_payload(uid, msg) # A missing/renamed script raises AutoControlJsonActionException (an # AutoControlException). Missing the base here let it escape *before* # the uid was marked seen below, so the same message re-fired every @@ -329,7 +333,7 @@ def _fire_for_uid(self, client: imaplib.IMAP4, trigger.trigger_id, error) else: trigger.last_error = None - trigger._seen_uids.add(uid_str) + trigger._seen_uids.add(uid) if trigger.mark_seen: _mark_seen(client, uid) return 1 diff --git a/je_auto_control/utils/triggers/trigger_engine.py b/je_auto_control/utils/triggers/trigger_engine.py index 60fe8d13..cf4d0ca7 100644 --- a/je_auto_control/utils/triggers/trigger_engine.py +++ b/je_auto_control/utils/triggers/trigger_engine.py @@ -12,7 +12,7 @@ import time import uuid from dataclasses import dataclass, field -from typing import Callable, Dict, List, Optional, Tuple +from typing import TYPE_CHECKING, Callable, Dict, List, Optional, Tuple from je_auto_control.utils.exception.exceptions import AutoControlException from je_auto_control.utils.json.json_file import read_action_json @@ -24,6 +24,9 @@ SOURCE_TRIGGER, STATUS_ERROR, STATUS_OK, default_history_store, ) +if TYPE_CHECKING: # imported lazily at runtime, where each trigger needs it + from je_auto_control.utils.scheduler.cron import CronExpression + @dataclass class _TriggerBase: @@ -177,7 +180,7 @@ class CronTrigger(_TriggerBase): only if the image is on screen". """ cron: str = "* * * * *" - _expr: Optional[object] = None + _expr: Optional["CronExpression"] = None _last_minute: Optional[str] = None def __post_init__(self) -> None: diff --git a/je_auto_control/utils/triggers/webhook_server.py b/je_auto_control/utils/triggers/webhook_server.py index 2d7b831b..8640df6c 100644 --- a/je_auto_control/utils/triggers/webhook_server.py +++ b/je_auto_control/utils/triggers/webhook_server.py @@ -238,10 +238,9 @@ def __init__(self, self._server: Optional[ThreadingHTTPServer] = None self._thread: Optional[threading.Thread] = None self._bound: Optional[Tuple[str, int]] = None - if executor is None: - self._executor = self._default_executor - else: - self._executor = executor + self._executor: Callable[[list, Dict[str, Any]], Any] = ( + self._default_executor if executor is None else executor + ) @property def is_running(self) -> bool: diff --git a/test/unit_test/headless/test_email_trigger.py b/test/unit_test/headless/test_email_trigger.py index 4ef7a772..cee05ea1 100644 --- a/test/unit_test/headless/test_email_trigger.py +++ b/test/unit_test/headless/test_email_trigger.py @@ -46,19 +46,24 @@ def select(self, mailbox, readonly=False): return ("OK", [b"1"]) def uid(self, command, *args): + # Normalise the way ``imaplib._command`` does before anything reaches + # the wire: skip ``None`` (the optional charset slot) and ASCII-encode + # ``str``. Pinning one call shape here made the stub, not the server, + # the thing the caller had to match. + parts = [arg.encode("ascii") if isinstance(arg, str) else arg + for arg in args if arg is not None] if command == "SEARCH": - criteria = args[1] - self.searches.append(criteria) + self.searches.append(parts[-1].decode("ascii")) return ("OK", [b" ".join(self._uids)]) if command == "FETCH": - uid = args[0] + uid = parts[0] self.fetched.append(uid) payload = self._messages.get(uid) if payload is None: return ("NO", [None]) return ("OK", [(b"1 (RFC822 {%d}" % len(payload), payload)]) if command == "STORE": - self.flagged.append(args[0]) + self.flagged.append(parts[0]) return ("OK", [b"stored"]) return ("NO", [None]) diff --git a/test/unit_test/headless/test_r3_net_triggers.py b/test/unit_test/headless/test_r3_net_triggers.py index 221e0058..eaa14af3 100644 --- a/test/unit_test/headless/test_r3_net_triggers.py +++ b/test/unit_test/headless/test_r3_net_triggers.py @@ -127,7 +127,7 @@ def missing_script(_path): trigger_id="t1", host="h", username="u", password="p", script_path="missing.json") - fired = watcher._fire_for_uid(client=object(), trigger=trigger, uid=b"42") + fired = watcher._fire_for_uid(client=object(), trigger=trigger, uid="42") assert fired == 1 assert "42" in trigger._seen_uids # processed -> no infinite re-fire diff --git a/test/verify/typing_contract_exempt.txt b/test/verify/typing_contract_exempt.txt index e4a1a94b..07fc94fc 100644 --- a/test/verify/typing_contract_exempt.txt +++ b/test/verify/typing_contract_exempt.txt @@ -14,7 +14,7 @@ # "Does not type-check" means on at least one of the three targets mypy can be # pointed at (win32, linux, darwin) — not just the one CI happens to run on. # -# Measured entries: 123 +# Measured entries: 108 je_auto_control.__main__ je_auto_control.cli @@ -32,11 +32,6 @@ je_auto_control.gui.recording_editor_tab je_auto_control.gui.remote_desktop.webrtc_dialogs je_auto_control.gui.remote_desktop.webrtc_panel je_auto_control.gui.test_suite_tab -je_auto_control.utils.accessibility.backends.base -je_auto_control.utils.accessibility.backends.linux_backend -je_auto_control.utils.accessibility.backends.windows_backend -je_auto_control.utils.accessibility.backends.windows_query -je_auto_control.utils.accessibility.recorder je_auto_control.utils.act_modes.act_modes je_auto_control.utils.actionability.actionability je_auto_control.utils.admin.admin_client @@ -46,7 +41,6 @@ je_auto_control.utils.annotate.annotate je_auto_control.utils.app_idle.app_idle je_auto_control.utils.bulkhead.bulkhead je_auto_control.utils.change_localize.change_localize -je_auto_control.utils.chatops.router je_auto_control.utils.clipboard.clipboard je_auto_control.utils.clipboard.win32_clipboard_api je_auto_control.utils.clipboard_formats.clipboard_formats @@ -63,7 +57,6 @@ je_auto_control.utils.dbus_client.session_bus je_auto_control.utils.deterministic.deterministic je_auto_control.utils.element_parse.element_parse je_auto_control.utils.element_proposal.element_proposal -je_auto_control.utils.element_repository.element_repository je_auto_control.utils.executor.action_executor je_auto_control.utils.expect_poll.expect_poll je_auto_control.utils.feature_match.feature_match @@ -82,14 +75,10 @@ je_auto_control.utils.locale_collation.locale_collation je_auto_control.utils.lock_session.lock_session je_auto_control.utils.mcp_server.__main__ je_auto_control.utils.mcp_server._client_requests -je_auto_control.utils.mcp_server.http_transport je_auto_control.utils.mcp_server.server je_auto_control.utils.mcp_server.tools._handlers je_auto_control.utils.monitor_layout.logical_frame je_auto_control.utils.mouse_path.mouse_path -je_auto_control.utils.observability.exporter -je_auto_control.utils.observability.metrics -je_auto_control.utils.observability.tracing je_auto_control.utils.ocr.backends je_auto_control.utils.plugin_loader.plugin_loader je_auto_control.utils.preprocess.preprocess @@ -100,7 +89,6 @@ je_auto_control.utils.rate_limit.rate_limit je_auto_control.utils.recording_edit.editor je_auto_control.utils.redaction.policies je_auto_control.utils.redaction.rules -je_auto_control.utils.rest_api.rest_server je_auto_control.utils.rotated_match.rotated_match je_auto_control.utils.run_history.history_store je_auto_control.utils.sbom.sbom @@ -117,9 +105,6 @@ je_auto_control.utils.text_normalize.text_normalize je_auto_control.utils.text_regions.text_regions je_auto_control.utils.tls_acme.challenge je_auto_control.utils.trash.trash -je_auto_control.utils.triggers.email_trigger -je_auto_control.utils.triggers.trigger_engine -je_auto_control.utils.triggers.webhook_server je_auto_control.utils.url_canon.url_canon je_auto_control.utils.usb.passthrough.key_provider je_auto_control.utils.usb.passthrough.session From edbf0861704a9fdaa328453d31298fd7a83e74c7 Mon Sep 17 00:00:00 2001 From: JeffreyChen Date: Sat, 22 Aug 2026 20:17:12 +0800 Subject: [PATCH 11/19] Settle the Win32 ctypes exemptions with 28 justified suppressions MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Re-measuring the DECIDE found it was twice the size it recorded and a different shape. Sixteen modules — not eight — pass on --platform win32 and fail on the other two targets for exactly one reason: typeshed declares windll / WinDLL / WINFUNCTYPE / WinError / get_last_error on Windows only. Half of them are nowhere near je_auto_control/windows/ (trash, app_idle, file_assoc, idle_keepawake, lock_session, session_guard, usb/passthrough/key_provider, gui/main_window), which rules out the option the entry had recommended: "measure a platform module on its own platform" cannot be a directory rule when half the affected modules are not in a platform directory. The maintainer chose per-site suppression. It came to 28 lines, not the 58 the entry projected — 58 counted each source line once per non-win32 target. Every one carries its own reason; none is blanket. Two things had to be measured. mypy honours `# type: ignore` only as the first comment on the line, so the two lines that already carried a `# nosec B607` take the marker first and merge the two justifications. And nine lines could not hold the marker inside the 120-char limit, so they are reformatted rather than shortened into meaninglessness: the marker follows an opening paren, and two sites hoist a value into a local first (`last_error` in the DPAPI wrapper, `kernel32` in the input hook), which reads better than the one-liner did. All sixteen were re-imported and exercised on Windows afterwards — a reformat only a type checker has verified is a reformat nobody has. --- Progress.md | 69 ++++++++----------- WHATS_NEW.md | 36 ++++++++++ architecture_explore.md | 22 +++--- je_auto_control/gui/main_window.py | 2 +- je_auto_control/utils/app_idle/app_idle.py | 2 +- .../utils/file_assoc/file_assoc.py | 2 +- .../utils/idle_keepawake/idle_keepawake.py | 2 +- .../utils/lock_session/lock_session.py | 2 +- .../utils/session_guard/session_guard.py | 2 +- je_auto_control/utils/trash/trash.py | 4 +- .../utils/usb/passthrough/key_provider.py | 15 ++-- .../windows/core/utils/win32_ctype_input.py | 4 +- .../core/utils/win32_keypress_check.py | 6 +- je_auto_control/windows/interception/_dll.py | 4 +- je_auto_control/windows/interception/mouse.py | 2 +- .../mouse/win32_ctype_mouse_control.py | 6 +- .../windows/record/win32_input_hook.py | 9 +-- .../windows/screen/win32_screen.py | 4 +- .../windows/window/windows_window_manage.py | 6 +- test/verify/typing_contract_exempt.txt | 18 +---- 20 files changed, 116 insertions(+), 101 deletions(-) diff --git a/Progress.md b/Progress.md index 63465ece..b5427eff 100644 --- a/Progress.md +++ b/Progress.md @@ -240,11 +240,11 @@ capability enum 值與 variadic `ei_seat_bind_capabilities`、event-type enum (`--cov-report=term-missing` 已經開著,CI 每一格都存了 `coverage.xml` artifact), 而不是齊頭式地補測試。 -### mypy:整包把關,108 個模組還沒過 +### mypy:整包把關,92 個模組還沒過 範圍不再是兩條路徑,而是**整包減去一張只准變少的清單** (`test/verify/typing_contract_exempt.txt`)。差別在於預設值:路徑清單只有人想到才會長, -新模組預設在圈外;現在新模組**預設就在契約裡**,1,017 個檔案有 909 個已經過關。 +新模組預設在圈外;現在新模組**預設就在契約裡**,1,017 個檔案有 925 個已經過關。 `wrapper` 那一群(6 個模組)在 2026-08-21 清掉了,做法見 [WHATS_NEW.md](WHATS_NEW.md): 平台縫的八個匯出名稱先宣告型別、再讓分支去綁定,其中三個用 @@ -300,10 +300,13 @@ accessibility 錯誤裡有三十四個是同一個沒標的回傳型別(`_unsu `_AtspiConnection._call` 在 `with` 之外沒有 bus 可以呼叫。 做法見 [WHATS_NEW.md](WHATS_NEW.md)。 -剩下 108 個模組要清。錯誤碼分布是 `attr-defined` 佔大宗,其次 +下面那條 `DECIDE` 也在同日拍板收掉,又清掉 16 個模組(做法見 +[WHATS_NEW.md](WHATS_NEW.md))。 + +剩下 92 個模組要清。錯誤碼分布是 `attr-defined` 佔大宗,其次 `arg-type`/`union-attr`/`assignment`,成群集中在 `gui`(11+`gui/remote_desktop` 2)、`utils/mcp_server`(4)、 -`utils/usb/passthrough`(4)、`utils/clipboard`(2),其餘散在各處。 +`utils/usb/passthrough`(3)、`utils/clipboard`(2),其餘散在各處。 以檔案計最大的三個是 `utils/usb/passthrough/winusb_backend.py`(34)、 `gui/_auto_click_tab.py`(31)、`gui/remote_desktop/webrtc_panel.py`(27)。 **下一步**是一次清一個群集,清完把行從清單刪掉—— @@ -336,50 +339,32 @@ accessibility 錯誤裡有三十四個是同一個沒標的回傳型別(`_unsu 真正的成本在後端那一側:四個平台的 `keyboard`/`mouse` 模組本身都還在豁免清單上, Protocol 一旦標上去,它們的內部型別錯誤就會一起浮出來。`linux_wayland`、 -`linux_with_x11`、`osx` 都已經清完,而 `windows/` 的**真實**型別錯誤也清完了 -(見上),只剩下面那條 `DECIDE` 的 ctypes 表面問題。所以前置條件實質上已經到位, +`linux_with_x11`、`osx` 都已經清完,而 `windows/` 在 2026-08-22 也全部離開了 +豁免清單(真實型別錯誤 + 下面那條已拍板的 ctypes 表面)。**前置條件已經全數到位**, 可以開始標 Protocol;下一次動這條的人不必再等別的群集。 -#### `DECIDE`:16 個模組的豁免不是它們的錯 - -**2026-08-22 重新實測,這一條的規模比原本寫的大一倍,而且形狀不同。** -原本寫的是「`windows/` 底下 8 個模組」;實際上是 **16 個模組、58 個錯誤**, -而其中**一半不在 `windows/` 底下**: +#### 已拍板(2026-08-22):Win32 ctypes 表面用 28 個逐行抑制解決 -| 模組 | 錯誤數 | -| --- | ---: | -| `windows/record/win32_input_hook.py` | 8 | -| `utils/usb/passthrough/key_provider.py` | 8 | -| `windows/core/utils/win32_ctype_input.py` | 6 | -| `windows/core/utils/win32_keypress_check.py`、`windows/interception/_dll.py`、`windows/mouse/win32_ctype_mouse_control.py`、`windows/screen/win32_screen.py`、`windows/window/windows_window_manage.py` | 各 4 | -| `windows/interception/mouse.py`、`gui/main_window.py`、`utils/app_idle/`、`utils/file_assoc/`、`utils/idle_keepawake/`、`utils/lock_session/`、`utils/session_guard/`、`utils/trash/` | 各 2 | +原本這裡是一條 `DECIDE`,寫的是「`windows/` 底下 8 個模組」。重新實測後是 +**16 個模組**,而且**一半不在 `windows/` 底下**(`utils/trash/`、`utils/app_idle/`、 +`utils/file_assoc/`、`utils/idle_keepawake/`、`utils/lock_session/`、 +`utils/session_guard/`、`utils/usb/passthrough/key_provider.py`、 +`gui/main_window.py`)——這一點直接否掉了原本推薦的那一條(照目錄決定用哪個平台量, +分不到這八個)。 -這 16 個模組在 `--platform win32` 上**全部乾淨**,在 linux/darwin 上的錯誤 -**100% 是同一件事**:`ctypes` 的 Win32 專屬表面(`windll`、`WinDLL`、 -`WINFUNCTYPE`、`WinError`、`get_last_error`)在 typeshed 的非 Windows 目標上 -根本沒有宣告。實測方式是把三個平台的結果對比,找出「win32 全綠、其他平台 -只錯在這組名字上」的模組——結果是 16 個,沒有一個是「順便還有別的錯」。 +**維護者選了逐行 `# type: ignore` 附理由**,實際只用了 **28 行**(原本估的 58 +是把同一行在 linux 與 darwin 各算了一次)。做法見 [WHATS_NEW.md](WHATS_NEW.md), +兩件必須實測的事記在這裡免得再踩: -**這件事會改動下面第四條的可行性**:`utils/app_idle/`、`utils/trash/`、 -`gui/main_window.py` 這些不在任何平台目錄底下,所以「照目錄決定用哪個平台量」 -的規則對它們無效。要嘛規則改成看模組內容(難自動判定),要嘛這八個走另一條路。 +* **mypy 只認每一行的第一個註解**——接在既有 `# nosec` 後面的 `# type: ignore` + 完全不生效(已實測)。所以有 `# nosec` 的那兩行,marker 放前面、兩個理由併成一句。 +* 有九行放不進 120 字元,是**改寫**而不是把理由砍到看不懂:括號換行時 marker 跟著 + 左括號走,兩處先把值取出來成區域變數(DPAPI 的 `last_error`、input hook 的 + `kernel32`),讀起來比原本的一行式更清楚。 -三條路都實測過,維護者挑一條: - -| 做法 | 成本 | 實測結果 | -| --- | --- | --- | -| **剪掉模組本體**(`if TYPE_CHECKING and sys.platform != "win32": raise`) | 每個檔 2 行 | **不能用**。mypy 一旦把模組本體判為 unreachable,`from ... import user32` 的那一端就會拿到 `Cannot determine type of "user32"` [has-type]——把 18 個錯誤換成一串新的、而且落在原本乾淨的模組上。 | -| **一個 shim 模組**把 Win32 專屬的 ctypes 表面集中匯出,非 Windows 分支綁 POSIX 對應物 | 新檔 + 18 個呼叫點改名 | 可行,事實只寫一次;代價是為了型別去動輸入熱路徑的 `user32` handle 建立方式。 | -| **18 個 `# type: ignore[attr-defined]`** 各附理由 | 18 行註記 | 可行,執行期零風險;代價是 18 個抑制,且同一句話重複 18 次。 | - -第四條是改閘門本身:讓 `typing_contract_verify.py` 只用模組自己的平台去量 -平台專屬模組(`windows/` 只用 win32、`osx/` 只用 darwin、`linux_*` 只用 linux)。 -(**但見上面那張表**:有八個受影響的模組不在平台目錄底下,照目錄分不到它們。) -這是三者裡最貼近事實的一條——拿 Linux 去檢查一個只跑得動在 Windows 的模組本來就沒有意義, -而閘門的 docstring 自己寫的多目標理由是「要讀得到平台分支後面的程式碼」,不是這個。 -但它會改掉「清單上的模組=在每個支援平台上都還沒過」這個既定語意,**所以要維護者拍板**。 - -推薦:第四條,其次是 shim。 +十六個模組事後都在真的 Windows 機器上重新 import 並實際呼叫過 +(`dpapi_available()`、`_windows_locked()`、`check_key_is_press`)—— +只有型別檢查器驗過的改寫等於沒人驗過。 有一件事別再踩:**這個閘門的判定不能隨環境浮動**。裝了 `[gui]`/`[webrtc]` 的開發機 與乾淨的 `pip install -e .` 曾經對 38 個模組看法不同(36 個 Qt 模組只在 PySide6 diff --git a/WHATS_NEW.md b/WHATS_NEW.md index fc033c4e..50008137 100644 --- a/WHATS_NEW.md +++ b/WHATS_NEW.md @@ -208,6 +208,42 @@ comparison as a `DECIDE`, because the cleanest of them changes what the gate means rather than what the code says. **136 modules left, 881 of 1,017 files inside the contract.** +### The Win32 ctypes DECIDE, Settled — and It Was Twice the Size It Said + +`Progress.md` carried a `DECIDE` about eight modules under +`je_auto_control/windows/` that pass on `--platform win32` and fail on the other +two targets for one reason: typeshed declares `windll`, `WinDLL`, `WINFUNCTYPE`, +`WinError` and `get_last_error` on Windows only. Re-measuring it — by diffing +the three platform runs and keeping the modules whose *entire* non-win32 error +set is that one surface — turned up **sixteen** modules, not eight, and half of +them are nowhere near `windows/`: `utils/trash/`, `utils/app_idle/`, +`utils/file_assoc/`, `utils/idle_keepawake/`, `utils/lock_session/`, +`utils/session_guard/`, `utils/usb/passthrough/key_provider.py` and +`gui/main_window.py`. That killed the option the entry had recommended — +"measure a platform module on its own platform" cannot be a directory rule when +half the affected modules are not in a platform directory. + +The maintainer picked the suppression route, and it came to **28 lines, not the +58 the entry projected**: 58 counted the same source line once for Linux and +once for macOS. Each carries its own reason, none is blanket, and the runtime is +untouched. + +Two things had to be measured rather than assumed. **mypy honours +`# type: ignore` only as the first comment on the line** — a trailing one after +an existing `# nosec` is silently ignored — so on the two lines that already had +a `# nosec B607` the marker goes first and the two justifications merge into one +`# reason:`. And nine lines could not hold the marker inside the 120-char limit, +so they were reformatted rather than shortened into meaninglessness: an opening +paren takes the comment (`ctypes.WinDLL( # type: ignore[…]`), and two sites +hoist a value into a local first — `last_error = ctypes.get_last_error()` in the +DPAPI wrapper, `kernel32 = ctypes.windll.kernel32` in the input hook — which +reads better than the one-liner did. + +All sixteen were re-imported and exercised on a real Windows machine afterwards +(`dpapi_available()`, `_windows_locked()`, `check_key_is_press`), because a +reformat that only a type checker verifies is a reformat nobody verified. +**92 modules left, 925 of 1,017 files inside the contract.** + ### Fifteen More Modules, and Four Errors That Were Wrong Rather Than Untyped The accessibility backends, the observability trio, the three triggers, diff --git a/architecture_explore.md b/architecture_explore.md index 087b411c..55b15197 100644 --- a/architecture_explore.md +++ b/architecture_explore.md @@ -20,7 +20,7 @@ iOS(WebDriverAgent)。核心能力是滑鼠/鍵盤控制、影像辨識、 | 指標 | 數值 | | --- | ---: | | Python 模組總數(含周邊子專案) | 1,031 | -| 程式碼總行數 | 140,680 | +| 程式碼總行數 | 140,690 | | `je_auto_control/utils/` 子套件數 | 310 | | `AC_*` 動作指令數(`known_commands()` 實測) | 773 | | 套件門面 `__all__` 公開名稱數 | 1,238 | @@ -183,7 +183,7 @@ socket server 有 8 MiB 讀取上限與 30 秒 handler timeout。 ### 5.3 平台後端 -#### Windows(`windows/`,23 檔/1,901 行) +#### Windows(`windows/`,23 檔/1,906 行) | 模組 | 行數 | 職責 | | --- | ---: | --- | @@ -510,7 +510,7 @@ socket server 有 8 MiB 讀取上限與 30 秒 handler timeout。 ### 5.4.10 遠端桌面與 USB -> 6 個套件、約 17,870 行。 +> 6 個套件、約 17,873 行。 | 模組 | 行數 | 職責 | | --- | ---: | --- | @@ -518,7 +518,7 @@ socket server 有 8 MiB 讀取上限與 30 秒 handler timeout。 | `utils/config_sync/` | 245 | 透過訊令伺服器做跨機器設定同步 | | `utils/device_matrix/` | 138 | 行動裝置矩陣:同一 action list 於多台裝置平行執行 | | `utils/remote_desktop/` | 11,990 | **遠端桌面子系統**(56 檔/11.7K LOC):TCP/WebSocket/WebRTC 三條傳輸路徑、主機與檢視端、訊令伺服器、TURN/中繼、多檢視者、錄影、信任清單、TOTP、稽核鏈 | -| `utils/usb/` | 4,250 | 跨平台 USB 列舉/熱插拔/裝置直通(WinUSB、IOKit、libusb 後端 + ACL + WebRTC DataChannel 通道) | +| `utils/usb/` | 4,253 | 跨平台 USB 列舉/熱插拔/裝置直通(WinUSB、IOKit、libusb 後端 + ACL + WebRTC DataChannel 通道) | | `utils/usbip/` | 920 | USB/IP 線路協定主機端(協定封包、TCP 伺服器、libusb URB 後端) | ### 5.4.11 伺服器、網路協定與外部整合 @@ -667,7 +667,7 @@ socket server 有 8 MiB 讀取上限與 30 秒 handler timeout。 ### 5.4.16 系統、視窗與剪貼簿 -> 16 個套件、約 2,403 行。 +> 16 個套件、約 2,405 行。 | 模組 | 行數 | 職責 | | --- | ---: | --- | @@ -682,7 +682,7 @@ socket server 有 8 MiB 讀取上限與 30 秒 handler timeout。 | `utils/rich_clipboard/` | 131 | 豐富剪貼簿格式 — HTML(CF_HTML)建構/解析/存取 | | `utils/shell_open/` | 97 | 以預設應用開啟檔案,或以預設瀏覽器開啟 URL | | `utils/system_volume/` | 194 | 讀取與控制系統主音量與靜音狀態 | -| `utils/trash/` | 88 | 把檔案移到系統資源回收筒(可復原刪除) | +| `utils/trash/` | 90 | 把檔案移到系統資源回收筒(可復原刪除) | | `utils/window_capture/` | 255 | 逐視窗截圖、視窗版面儲存/還原、貼齊與排列 | | `utils/window_geometry/` | 81 | 視窗客戶區幾何(外框內縮、client→screen 對映) | | `utils/window_layout/` | 134 | 視窗拼貼/版面規劃器(左右半、四象限、網格、層疊) | @@ -777,7 +777,7 @@ socket server 有 8 MiB 讀取上限與 30 秒 handler timeout。 | `permissions.py` / `clipboard_sync.py` / `wake_on_lan.py` / `session_actions.py` / `auth.py` | 65 / 73 / 57 / 41 / 29 | 逐 session 權限、剪貼簿同步、WOL、SAS 注入與螢幕遮蔽、HMAC 挑戰回應。 | | `ws_host.py` / `ws_viewer.py` / `jpeg_recorder.py` | 41 / 30 / 139 | WebSocket 傳輸變體與 TCP 路徑錄影。 | -#### `utils/usb/`(4,250 行)與 `utils/usbip/`(920 行) +#### `utils/usb/`(4,253 行)與 `utils/usbip/`(920 行) | 檔案 | 行數 | 職責 | | --- | ---: | --- | @@ -1028,11 +1028,11 @@ socket 預設綁 `127.0.0.1`;資源一律用 `with`。 | `utils/mcp_server/` | 21 | 17,330 | | `utils/remote_desktop/` | 56 | 11,990 | | `utils/executor/` | 6 | 9,075 | -| `utils/usb/` | 17 | 4,250 | +| `utils/usb/` | 17 | 4,253 | | `je_auto_control/`(頂層 3 檔) | 3 | 2,363 | | `utils/accessibility/` | 13 | 2,835 | | `wrapper/` | 19 | 3,293 | -| `windows/` | 23 | 1,901 | +| `windows/` | 23 | 1,906 | | `utils/rest_api/` | 8 | 1,751 | | `utils/agent/` | 8 | 1,250 | | `linux_with_x11/` | 19 | 1,236 | @@ -1044,6 +1044,6 @@ socket 預設綁 `127.0.0.1`;資源一律用 `with`。 | `osx/` | 17 | 915 | | `autocontrol-lsp/` | 8 | 744 | | `utils/hotkey/` | 7 | 727 | -| 其餘模組(約 286 個 `utils/` 子套件 + `android/`/`ios/`/周邊小工具) | 673 | 47,496 | -| **總計** | **1,025** | **140,615** | +| 其餘模組(約 286 個 `utils/` 子套件 + `android/`/`ios/`/周邊小工具) | 673 | 47,498 | +| **總計** | **1,025** | **140,625** | diff --git a/je_auto_control/gui/main_window.py b/je_auto_control/gui/main_window.py index 863f5424..7a8423c3 100644 --- a/je_auto_control/gui/main_window.py +++ b/je_auto_control/gui/main_window.py @@ -42,7 +42,7 @@ def __init__(self) -> None: super().__init__() self.app_id = _t("application_name", "AutoControlGUI") if sys.platform in ["win32", "cygwin", "msys"]: - from ctypes import windll + from ctypes import windll # type: ignore[attr-defined] # reason: win32-only ctypes windll.shell32.SetCurrentProcessExplicitAppUserModelID(self.app_id) self._user_font_pt: int = 0 # 0 means auto-detect from screen diff --git a/je_auto_control/utils/app_idle/app_idle.py b/je_auto_control/utils/app_idle/app_idle.py index e42528ec..acd7d0cf 100644 --- a/je_auto_control/utils/app_idle/app_idle.py +++ b/je_auto_control/utils/app_idle/app_idle.py @@ -84,7 +84,7 @@ class _CURSORINFO(ctypes.Structure): _fields_ = [("cbSize", ctypes.c_uint), ("flags", ctypes.c_uint), ("hCursor", ctypes.c_void_p), ("ptScreenPos", _POINT)] - user32 = ctypes.windll.user32 + user32 = ctypes.windll.user32 # type: ignore[attr-defined] # reason: win32-only ctypes info = _CURSORINFO() info.cbSize = ctypes.sizeof(_CURSORINFO) if not user32.GetCursorInfo(ctypes.byref(info)): diff --git a/je_auto_control/utils/file_assoc/file_assoc.py b/je_auto_control/utils/file_assoc/file_assoc.py index 245827d3..5e0319be 100644 --- a/je_auto_control/utils/file_assoc/file_assoc.py +++ b/je_auto_control/utils/file_assoc/file_assoc.py @@ -48,7 +48,7 @@ def normalize_ext(target: str) -> str: def _assoc_query(ext: str, assoc_str: int) -> Optional[str]: """Run one AssocQueryStringW lookup; return the string or None.""" - shlwapi = ctypes.windll.shlwapi + shlwapi = ctypes.windll.shlwapi # type: ignore[attr-defined] # reason: win32-only ctypes size = ctypes.c_ulong(0) shlwapi.AssocQueryStringW(0, assoc_str, ext, None, None, ctypes.byref(size)) if size.value == 0: diff --git a/je_auto_control/utils/idle_keepawake/idle_keepawake.py b/je_auto_control/utils/idle_keepawake/idle_keepawake.py index 9c5b0700..95203be5 100644 --- a/je_auto_control/utils/idle_keepawake/idle_keepawake.py +++ b/je_auto_control/utils/idle_keepawake/idle_keepawake.py @@ -109,7 +109,7 @@ def plan_keep_awake(*, display: bool = True, def _win_keep_awake(flags: int) -> Callable[[], None]: - kernel32 = ctypes.windll.kernel32 + kernel32 = ctypes.windll.kernel32 # type: ignore[attr-defined] # reason: win32-only ctypes kernel32.SetThreadExecutionState(ctypes.c_uint(flags)) def _release() -> None: diff --git a/je_auto_control/utils/lock_session/lock_session.py b/je_auto_control/utils/lock_session/lock_session.py index cacca6aa..be96acdc 100644 --- a/je_auto_control/utils/lock_session/lock_session.py +++ b/je_auto_control/utils/lock_session/lock_session.py @@ -32,7 +32,7 @@ def _win_lock() -> bool: """Lock the Windows workstation via ``LockWorkStation``.""" import ctypes - user32 = ctypes.windll.user32 # nosec B607 # reason: fixed system DLL + user32 = ctypes.windll.user32 # type: ignore[attr-defined] # nosec B607 # reason: win32-only ctypes, fixed DLL return bool(user32.LockWorkStation()) diff --git a/je_auto_control/utils/session_guard/session_guard.py b/je_auto_control/utils/session_guard/session_guard.py index 202bedb8..242b671c 100644 --- a/je_auto_control/utils/session_guard/session_guard.py +++ b/je_auto_control/utils/session_guard/session_guard.py @@ -20,7 +20,7 @@ def _windows_locked() -> bool: """True when the Windows input desktop can't be opened (station locked).""" import ctypes _DESKTOP_READOBJECTS = 0x0001 - user32 = ctypes.windll.user32 # nosec B607 # reason: fixed system DLL + user32 = ctypes.windll.user32 # type: ignore[attr-defined] # nosec B607 # reason: win32-only ctypes, fixed DLL handle = user32.OpenInputDesktop(0, False, _DESKTOP_READOBJECTS) if not handle: return True diff --git a/je_auto_control/utils/trash/trash.py b/je_auto_control/utils/trash/trash.py index 2ab844a7..8bae1254 100644 --- a/je_auto_control/utils/trash/trash.py +++ b/je_auto_control/utils/trash/trash.py @@ -49,7 +49,9 @@ class _SHFILEOPSTRUCTW(ctypes.Structure): operation.wFunc = fo_delete operation.pFrom = path + "\0\0" # the path list is double-null terminated operation.fFlags = flags - result = ctypes.windll.shell32.SHFileOperationW(ctypes.byref(operation)) + result = ctypes.windll.shell32.SHFileOperationW( # type: ignore[attr-defined] # reason: win32-only ctypes + ctypes.byref(operation), + ) if result != 0: raise OSError(f"SHFileOperation failed ({result}) for {path!r}") diff --git a/je_auto_control/utils/usb/passthrough/key_provider.py b/je_auto_control/utils/usb/passthrough/key_provider.py index d4b03484..6a76916e 100644 --- a/je_auto_control/utils/usb/passthrough/key_provider.py +++ b/je_auto_control/utils/usb/passthrough/key_provider.py @@ -34,7 +34,7 @@ def dpapi_available() -> bool: return False try: import ctypes - ctypes.WinDLL("crypt32") + ctypes.WinDLL("crypt32") # type: ignore[attr-defined] # reason: win32-only ctypes return True except OSError: return False @@ -48,8 +48,12 @@ class _Blob(ctypes.Structure): _fields_ = [("cbData", wintypes.DWORD), ("pbData", ctypes.POINTER(ctypes.c_char))] - crypt32 = ctypes.WinDLL("crypt32", use_last_error=True) - kernel32 = ctypes.WinDLL("kernel32", use_last_error=True) + crypt32 = ctypes.WinDLL( # type: ignore[attr-defined] # reason: win32-only ctypes + "crypt32", use_last_error=True, + ) + kernel32 = ctypes.WinDLL( # type: ignore[attr-defined] # reason: win32-only ctypes + "kernel32", use_last_error=True, + ) func = getattr(crypt32, func_name) func.restype = wintypes.BOOL @@ -61,9 +65,8 @@ class _Blob(ctypes.Structure): _CRYPTPROTECT_UI_FORBIDDEN, ctypes.byref(out), ) if not ok: - raise RuntimeError( - f"{func_name} failed: {ctypes.get_last_error()}", - ) + last_error = ctypes.get_last_error() # type: ignore[attr-defined] # reason: win32-only ctypes + raise RuntimeError(f"{func_name} failed: {last_error}") try: return ctypes.string_at(out.pbData, out.cbData) finally: diff --git a/je_auto_control/windows/core/utils/win32_ctype_input.py b/je_auto_control/windows/core/utils/win32_ctype_input.py index ac292a8f..29472d23 100644 --- a/je_auto_control/windows/core/utils/win32_ctype_input.py +++ b/je_auto_control/windows/core/utils/win32_ctype_input.py @@ -10,7 +10,7 @@ from ctypes import wintypes from je_auto_control.windows.core.utils.win32_vk import WIN32_EventF_UNICODE, WIN32_VkToVSC -user32 = ctypes.WinDLL('user32', use_last_error=True) +user32 = ctypes.WinDLL('user32', use_last_error=True) # type: ignore[attr-defined] # reason: win32-only ctypes Mouse: int = 0 Keyboard: int = 1 @@ -58,7 +58,7 @@ class INPUTUnion(ctypes.Union): def _check_count(result, func, args) -> list: if result == 0: - raise ctypes.WinError(ctypes.get_last_error()) + raise ctypes.WinError(ctypes.get_last_error()) # type: ignore[attr-defined] # reason: win32-only ctypes return args diff --git a/je_auto_control/windows/core/utils/win32_keypress_check.py b/je_auto_control/windows/core/utils/win32_keypress_check.py index 4162781a..e835e5fd 100644 --- a/je_auto_control/windows/core/utils/win32_keypress_check.py +++ b/je_auto_control/windows/core/utils/win32_keypress_check.py @@ -12,9 +12,11 @@ def check_key_is_press(keycode: Union[int, str]) -> bool: if isinstance(keycode, int): - temp: int = ctypes.windll.user32.GetAsyncKeyState(keycode) + temp: int = ctypes.windll.user32.GetAsyncKeyState( # type: ignore[attr-defined] # reason: win32-only ctypes + keycode) else: - temp = ctypes.windll.user32.GetAsyncKeyState(ord(keycode)) + temp = ctypes.windll.user32.GetAsyncKeyState( # type: ignore[attr-defined] # reason: win32-only ctypes + ord(keycode)) if temp != 0: return True return False diff --git a/je_auto_control/windows/interception/_dll.py b/je_auto_control/windows/interception/_dll.py index 35139ed8..36217fe9 100644 --- a/je_auto_control/windows/interception/_dll.py +++ b/je_auto_control/windows/interception/_dll.py @@ -92,7 +92,7 @@ def _load_dll() -> ctypes.CDLL: """Load and prototype ``interception.dll``.""" path = _resolve_dll_path() try: - dll = ctypes.WinDLL(path) + dll = ctypes.WinDLL(path) # type: ignore[attr-defined] # reason: win32-only ctypes except OSError as exc: raise InterceptionUnavailable( f"could not load {path!r}: {exc}. Install the driver from " @@ -192,7 +192,7 @@ def default_mouse_device() -> int: # --- vk → scancode helper --------------------------------------------------- -_user32 = ctypes.WinDLL("user32", use_last_error=True) +_user32 = ctypes.WinDLL("user32", use_last_error=True) # type: ignore[attr-defined] # reason: win32-only ctypes _user32.MapVirtualKeyW.restype = wintypes.UINT _user32.MapVirtualKeyW.argtypes = [wintypes.UINT, wintypes.UINT] diff --git a/je_auto_control/windows/interception/mouse.py b/je_auto_control/windows/interception/mouse.py index 6fba5328..e3eb9fa3 100644 --- a/je_auto_control/windows/interception/mouse.py +++ b/je_auto_control/windows/interception/mouse.py @@ -15,7 +15,7 @@ import ctypes import sys -from ctypes import windll, wintypes +from ctypes import windll, wintypes # type: ignore[attr-defined] # reason: win32-only ctypes from typing import Optional, Tuple from je_auto_control.utils.exception.exception_tags import ( diff --git a/je_auto_control/windows/mouse/win32_ctype_mouse_control.py b/je_auto_control/windows/mouse/win32_ctype_mouse_control.py index 8f2bcd86..463c5339 100644 --- a/je_auto_control/windows/mouse/win32_ctype_mouse_control.py +++ b/je_auto_control/windows/mouse/win32_ctype_mouse_control.py @@ -1,6 +1,6 @@ import sys from typing import Tuple, Optional -from ctypes import windll +from ctypes import windll # type: ignore[attr-defined] # reason: win32-only ctypes from je_auto_control.utils.exception.exception_tags import windows_import_error_message from je_auto_control.utils.exception.exceptions import AutoControlException @@ -99,7 +99,7 @@ def mouse_event(event: int, x: int, y: int, dw_data: int = 0) -> None: :param dw_data: 滾輪數值 Wheel data """ converted_x, converted_y = _convert_position(x, y) - ctypes.windll.user32.mouse_event( + ctypes.windll.user32.mouse_event( # type: ignore[attr-defined] # reason: win32-only ctypes event, ctypes.c_long(converted_x), ctypes.c_long(converted_y), @@ -217,4 +217,4 @@ def send_mouse_event_to_window(window, mouse_keycode: Tuple[int, int, int], lparam = (int(y) << 16) | int(x) (down_msg, down_wparam), (up_msg, up_wparam) = _resolve_window_messages(mouse_keycode) user32.PostMessageW(window, down_msg, down_wparam, lparam) - user32.PostMessageW(window, up_msg, up_wparam, lparam) \ No newline at end of file + user32.PostMessageW(window, up_msg, up_wparam, lparam) diff --git a/je_auto_control/windows/record/win32_input_hook.py b/je_auto_control/windows/record/win32_input_hook.py index d2f171f4..48ca3307 100644 --- a/je_auto_control/windows/record/win32_input_hook.py +++ b/je_auto_control/windows/record/win32_input_hook.py @@ -101,7 +101,7 @@ def stop(self) -> List[Dict[str, Any]]: """Stop recording and return the raw events.""" if self._thread_id: try: - ctypes.windll.user32.PostThreadMessageW( + ctypes.windll.user32.PostThreadMessageW( # type: ignore[attr-defined] # reason: win32-only ctypes self._thread_id, WM_QUIT, 0, 0) except OSError as error: autocontrol_logger.error("recorder stop failed: %r", error) @@ -109,7 +109,7 @@ def stop(self) -> List[Dict[str, Any]]: # -- hook thread ------------------------------------------------------- def _run(self) -> None: - user32 = ctypes.windll.user32 + user32 = ctypes.windll.user32 # type: ignore[attr-defined] # reason: win32-only ctypes try: self._install(user32) except OSError as error: @@ -128,8 +128,9 @@ def _run(self) -> None: self._unhook(user32) def _install(self, user32) -> None: - self._thread_id = ctypes.windll.kernel32.GetCurrentThreadId() - proto = ctypes.WINFUNCTYPE( + kernel32 = ctypes.windll.kernel32 # type: ignore[attr-defined] # reason: win32-only ctypes + self._thread_id = kernel32.GetCurrentThreadId() + proto = ctypes.WINFUNCTYPE( # type: ignore[attr-defined] # reason: win32-only ctypes ctypes.c_ssize_t, ctypes.c_int, wintypes.WPARAM, wintypes.LPARAM) # Declare the signatures: a hook handle is 64-bit and the default # c_int return would truncate it. diff --git a/je_auto_control/windows/screen/win32_screen.py b/je_auto_control/windows/screen/win32_screen.py index 2e96a65d..52d36e55 100644 --- a/je_auto_control/windows/screen/win32_screen.py +++ b/je_auto_control/windows/screen/win32_screen.py @@ -18,8 +18,8 @@ # This module owns its user32 / gdi32 handles rather than sharing # ``ctypes.windll``: prototypes live on the function objects, so a shared handle # would leak these declarations into every other caller in the process. -_user32 = ctypes.WinDLL("user32", use_last_error=True) -_gdi32 = ctypes.WinDLL("gdi32", use_last_error=True) +_user32 = ctypes.WinDLL("user32", use_last_error=True) # type: ignore[attr-defined] # reason: win32-only ctypes +_gdi32 = ctypes.WinDLL("gdi32", use_last_error=True) # type: ignore[attr-defined] # reason: win32-only ctypes # HDC 是**指標寬度**的 handle。ctypes 預設把回傳值與參數當成 c_int,在 64 位元 # Windows 上會截斷——`GetDC` 回來就已經是壞的,再傳給 `GetPixel` / `ReleaseDC` diff --git a/je_auto_control/windows/window/windows_window_manage.py b/je_auto_control/windows/window/windows_window_manage.py index 5ea10eb8..4eb6baf8 100644 --- a/je_auto_control/windows/window/windows_window_manage.py +++ b/je_auto_control/windows/window/windows_window_manage.py @@ -3,7 +3,9 @@ Windows window management (Win32 ctypes) """ import ctypes -from ctypes import WINFUNCTYPE, byref, create_unicode_buffer, wintypes +from ctypes import ( # type: ignore[attr-defined] # reason: win32-only ctypes + WINFUNCTYPE, byref, create_unicode_buffer, wintypes, +) from typing import List, Optional, Tuple # 相容用途:舊版本從這個模組匯出共用的 user32。 @@ -18,7 +20,7 @@ # This module deliberately owns its own user32 handle: argtypes/restype live on # the function objects, so sharing one would leak these prototypes into other # callers (`utils/window_capture/` passes its own RECT to GetWindowRect). -_user32 = ctypes.WinDLL("user32", use_last_error=True) +_user32 = ctypes.WinDLL("user32", use_last_error=True) # type: ignore[attr-defined] # reason: win32-only ctypes # HWND 是指標寬度的 handle。ctypes 預設把參數與回傳值當成 c_int,在 64 位元 # Windows 上會截斷成 32 位元——與 `OpenProcess` 那個經典陷阱同一類。每個函式 diff --git a/test/verify/typing_contract_exempt.txt b/test/verify/typing_contract_exempt.txt index 07fc94fc..74204883 100644 --- a/test/verify/typing_contract_exempt.txt +++ b/test/verify/typing_contract_exempt.txt @@ -14,7 +14,7 @@ # "Does not type-check" means on at least one of the three targets mypy can be # pointed at (win32, linux, darwin) — not just the one CI happens to run on. # -# Measured entries: 108 +# Measured entries: 92 je_auto_control.__main__ je_auto_control.cli @@ -27,7 +27,6 @@ je_auto_control.gui._script_tab je_auto_control.gui.assertions_tab je_auto_control.gui.language_wrapper.multi_language_wrapper je_auto_control.gui.main_widget -je_auto_control.gui.main_window je_auto_control.gui.recording_editor_tab je_auto_control.gui.remote_desktop.webrtc_dialogs je_auto_control.gui.remote_desktop.webrtc_panel @@ -38,7 +37,6 @@ je_auto_control.utils.admin.admin_client je_auto_control.utils.agent_memory.agent_memory je_auto_control.utils.anchor_locator.locator je_auto_control.utils.annotate.annotate -je_auto_control.utils.app_idle.app_idle je_auto_control.utils.bulkhead.bulkhead je_auto_control.utils.change_localize.change_localize je_auto_control.utils.clipboard.clipboard @@ -60,7 +58,6 @@ je_auto_control.utils.element_proposal.element_proposal je_auto_control.utils.executor.action_executor je_auto_control.utils.expect_poll.expect_poll je_auto_control.utils.feature_match.feature_match -je_auto_control.utils.file_assoc.file_assoc je_auto_control.utils.flake_cluster.flake_cluster je_auto_control.utils.flow_debugger.flow_debugger je_auto_control.utils.generate_report.generate_xml_report @@ -69,10 +66,8 @@ je_auto_control.utils.hotkey.backends.linux_backend je_auto_control.utils.hotkey.backends.windows_backend je_auto_control.utils.http_client.http_client je_auto_control.utils.humanize.motion -je_auto_control.utils.idle_keepawake.idle_keepawake je_auto_control.utils.json_patch.json_patch je_auto_control.utils.locale_collation.locale_collation -je_auto_control.utils.lock_session.lock_session je_auto_control.utils.mcp_server.__main__ je_auto_control.utils.mcp_server._client_requests je_auto_control.utils.mcp_server.server @@ -96,7 +91,6 @@ je_auto_control.utils.scheduler.scheduler je_auto_control.utils.script_vars.interpolate je_auto_control.utils.search_index.search_index je_auto_control.utils.secrets.secret_store -je_auto_control.utils.session_guard.session_guard je_auto_control.utils.socket_server.auto_control_socket_server je_auto_control.utils.soft_assert.soft_assert je_auto_control.utils.table_grid_fill.table_grid_fill @@ -104,9 +98,7 @@ je_auto_control.utils.test_record.record_test_class je_auto_control.utils.text_normalize.text_normalize je_auto_control.utils.text_regions.text_regions je_auto_control.utils.tls_acme.challenge -je_auto_control.utils.trash.trash je_auto_control.utils.url_canon.url_canon -je_auto_control.utils.usb.passthrough.key_provider je_auto_control.utils.usb.passthrough.session je_auto_control.utils.usb.passthrough.viewer_client je_auto_control.utils.usb.passthrough.winusb_backend @@ -116,11 +108,3 @@ je_auto_control.utils.window_capture.window_capture je_auto_control.utils.window_zorder.window_zorder je_auto_control.utils.work_queue.work_queue je_auto_control.utils.xml.xml_file.xml_file -je_auto_control.windows.core.utils.win32_ctype_input -je_auto_control.windows.core.utils.win32_keypress_check -je_auto_control.windows.interception._dll -je_auto_control.windows.interception.mouse -je_auto_control.windows.mouse.win32_ctype_mouse_control -je_auto_control.windows.record.win32_input_hook -je_auto_control.windows.screen.win32_screen -je_auto_control.windows.window.windows_window_manage From 283e320d51efc11fc392a5db139e4dd61e69b647 Mon Sep 17 00:00:00 2001 From: JeffreyChen Date: Sat, 22 Aug 2026 20:32:37 +0800 Subject: [PATCH 12/19] Clear the WinUSB and clipboard clusters, and fix what they were hiding MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Both were the same mistake told two ways — a handle whose declared type cannot do what the code asks of it — and both hid a real defect. `winusb_backend._load_dlls` published its three DLL handles one at a time behind `if _setupapi is not None: return`. A failure loading winusb.dll — what happens on a machine with no WinUSB-bound device — left `_setupapi` set, so the guard short-circuited every later attempt and the call sites got `AttributeError: 'NoneType' object has no attribute 'WinUsb_Initialize'` instead of the retry the guard exists to allow. The three now come back from one loader as a NamedTuple, published only after all three load. With them: a device enumerated without an interface path is skipped rather than passed to CreateFileW as None, and a WinUsb_Initialize that reports success with a null handle is a failure rather than an Optional[int] handed to the wrapper. `clipboard_api()` returned `Tuple[object, object]`. `object` has no attributes, so all twelve calls through it were errors — in the one module whose whole docstring is about declaring these prototypes correctly. A ctypes library resolves symbols through `__getattr__`, so `Any` is the only honest promise and is what it says now. Both were exercised against the real thing on Windows: a clipboard text round trip, `clipboard_formats()` on a live clipboard, and the WinUSB backend enumerating an actual bound device. --- Progress.md | 23 ++-- WHATS_NEW.md | 33 +++++ architecture_explore.md | 18 +-- je_auto_control/utils/clipboard/clipboard.py | 16 +-- .../utils/clipboard/win32_clipboard_api.py | 17 ++- .../utils/usb/passthrough/session.py | 2 +- .../utils/usb/passthrough/viewer_client.py | 2 +- .../utils/usb/passthrough/winusb_backend.py | 114 +++++++++++------- .../headless/test_usb_platform_backends.py | 9 +- test/verify/typing_contract_exempt.txt | 8 +- 10 files changed, 156 insertions(+), 86 deletions(-) diff --git a/Progress.md b/Progress.md index b5427eff..92b6eacd 100644 --- a/Progress.md +++ b/Progress.md @@ -240,11 +240,11 @@ capability enum 值與 variadic `ei_seat_bind_capabilities`、event-type enum (`--cov-report=term-missing` 已經開著,CI 每一格都存了 `coverage.xml` artifact), 而不是齊頭式地補測試。 -### mypy:整包把關,92 個模組還沒過 +### mypy:整包把關,86 個模組還沒過 範圍不再是兩條路徑,而是**整包減去一張只准變少的清單** (`test/verify/typing_contract_exempt.txt`)。差別在於預設值:路徑清單只有人想到才會長, -新模組預設在圈外;現在新模組**預設就在契約裡**,1,017 個檔案有 925 個已經過關。 +新模組預設在圈外;現在新模組**預設就在契約裡**,1,017 個檔案有 931 個已經過關。 `wrapper` 那一群(6 個模組)在 2026-08-21 清掉了,做法見 [WHATS_NEW.md](WHATS_NEW.md): 平台縫的八個匯出名稱先宣告型別、再讓分支去綁定,其中三個用 @@ -303,12 +303,19 @@ accessibility 錯誤裡有三十四個是同一個沒標的回傳型別(`_unsu 下面那條 `DECIDE` 也在同日拍板收掉,又清掉 16 個模組(做法見 [WHATS_NEW.md](WHATS_NEW.md))。 -剩下 92 個模組要清。錯誤碼分布是 `attr-defined` 佔大宗,其次 -`arg-type`/`union-attr`/`assignment`,成群集中在 -`gui`(11+`gui/remote_desktop` 2)、`utils/mcp_server`(4)、 -`utils/usb/passthrough`(3)、`utils/clipboard`(2),其餘散在各處。 -以檔案計最大的三個是 `utils/usb/passthrough/winusb_backend.py`(34)、 -`gui/_auto_click_tab.py`(31)、`gui/remote_desktop/webrtc_panel.py`(27)。 +ctypes 表面一收掉,卡在它後面的兩群也跟著清了:`utils/usb/passthrough`(3) +與 `utils/clipboard`(3)。兩邊都藏著真的缺陷——`winusb_backend._load_dlls` +一次公布一個 handle,第二個載入失敗時第一個已經設好,於是 `is not None` 的守衛 +把之後每一次重試都短路掉;`clipboard_api()` 宣告回傳 `Tuple[object, object]`, +而 `object` 沒有任何屬性,所以透過它的十二個 Win32 呼叫全是型別錯誤。 +做法見 [WHATS_NEW.md](WHATS_NEW.md)。 + +剩下 86 個模組要清。錯誤碼分布是 `attr-defined` 佔大宗,其次 +`arg-type`/`union-attr`/`assignment`,**最大的一群現在是 `gui`** +(11+`gui/remote_desktop` 2,共 89 個錯誤),其次 `utils/mcp_server`(4,38 個)。 +以檔案計最大的三個是 `gui/_auto_click_tab.py`(31)、 +`gui/remote_desktop/webrtc_panel.py`(27)、 +`utils/mcp_server/_client_requests.py`(21)。 **下一步**是一次清一個群集,清完把行從清單刪掉—— `python test/verify/typing_contract_verify.py --fix` 會替你改,CI 會在你忘了刪的時候紅掉。 已經清掉的那些留下四個可以直接套用的樣板:mixin 用 `if TYPE_CHECKING:` 宣告 diff --git a/WHATS_NEW.md b/WHATS_NEW.md index 50008137..d7488c0f 100644 --- a/WHATS_NEW.md +++ b/WHATS_NEW.md @@ -208,6 +208,39 @@ comparison as a `DECIDE`, because the cleanest of them changes what the gate means rather than what the code says. **136 modules left, 881 of 1,017 files inside the contract.** +### The WinUSB Backend Was One Failed DLL Load Away From Never Recovering + +With the ctypes surface settled, the two clusters behind it came off: +`utils/usb/passthrough` and `utils/clipboard`. Both were the same mistake told +two ways — a handle whose declared type could not do what the code asks of it — +and both hid a real defect behind it. + +**`winusb_backend` published its three DLL handles one at a time.** `_load_dlls` +assigned `_setupapi`, then `_winusb`, then `_kernel32`, guarded by +`if _setupapi is not None: return`. If loading `winusb.dll` raised — which is +exactly what happens on a machine where no device is bound to WinUSB and the +DLL is absent — `_setupapi` was already set, so the guard short-circuited every +later attempt and every call site got +`AttributeError: 'NoneType' object has no attribute 'WinUsb_Initialize'` +instead of the retry the guard was written to allow. The three handles now come +back from one loader as a `NamedTuple`, published only after all three load. +Two smaller ones went with it: a device enumerated without an interface path is +skipped rather than passed to `CreateFileW` as `None`, and a `WinUsb_Initialize` +that reports success with a null handle is now a failure rather than an +`Optional[int]` handed to the handle wrapper. + +**`clipboard_api()` returned `Tuple[object, object]`.** `object` has no +attributes, so all twelve `user32.OpenClipboard` / `kernel32.GlobalLock` calls +through it were type errors — on a module whose entire docstring is about +getting these prototypes right once. A ctypes library resolves every symbol +through `__getattr__`, so `Any` is the only honest promise, and it is what the +signature says now. + +Both were exercised against the real thing on Windows afterwards: a clipboard +text round trip, `clipboard_formats()` against a live clipboard, and the WinUSB +backend enumerating an actual bound device. **86 modules left, 931 of 1,017 +files inside the contract.** + ### The Win32 ctypes DECIDE, Settled — and It Was Twice the Size It Said `Progress.md` carried a `DECIDE` about eight modules under diff --git a/architecture_explore.md b/architecture_explore.md index 55b15197..1c55de13 100644 --- a/architecture_explore.md +++ b/architecture_explore.md @@ -20,7 +20,7 @@ iOS(WebDriverAgent)。核心能力是滑鼠/鍵盤控制、影像辨識、 | 指標 | 數值 | | --- | ---: | | Python 模組總數(含周邊子專案) | 1,031 | -| 程式碼總行數 | 140,690 | +| 程式碼總行數 | 140,725 | | `je_auto_control/utils/` 子套件數 | 310 | | `AC_*` 動作指令數(`known_commands()` 實測) | 773 | | 套件門面 `__all__` 公開名稱數 | 1,238 | @@ -510,7 +510,7 @@ socket server 有 8 MiB 讀取上限與 30 秒 handler timeout。 ### 5.4.10 遠端桌面與 USB -> 6 個套件、約 17,873 行。 +> 6 個套件、約 17,901 行。 | 模組 | 行數 | 職責 | | --- | ---: | --- | @@ -518,7 +518,7 @@ socket server 有 8 MiB 讀取上限與 30 秒 handler timeout。 | `utils/config_sync/` | 245 | 透過訊令伺服器做跨機器設定同步 | | `utils/device_matrix/` | 138 | 行動裝置矩陣:同一 action list 於多台裝置平行執行 | | `utils/remote_desktop/` | 11,990 | **遠端桌面子系統**(56 檔/11.7K LOC):TCP/WebSocket/WebRTC 三條傳輸路徑、主機與檢視端、訊令伺服器、TURN/中繼、多檢視者、錄影、信任清單、TOTP、稽核鏈 | -| `utils/usb/` | 4,253 | 跨平台 USB 列舉/熱插拔/裝置直通(WinUSB、IOKit、libusb 後端 + ACL + WebRTC DataChannel 通道) | +| `utils/usb/` | 4,281 | 跨平台 USB 列舉/熱插拔/裝置直通(WinUSB、IOKit、libusb 後端 + ACL + WebRTC DataChannel 通道) | | `utils/usbip/` | 920 | USB/IP 線路協定主機端(協定封包、TCP 伺服器、libusb URB 後端) | ### 5.4.11 伺服器、網路協定與外部整合 @@ -667,11 +667,11 @@ socket server 有 8 MiB 讀取上限與 30 秒 handler timeout。 ### 5.4.16 系統、視窗與剪貼簿 -> 16 個套件、約 2,405 行。 +> 16 個套件、約 2,412 行。 | 模組 | 行數 | 職責 | | --- | ---: | --- | -| `utils/clipboard/` | 489 | 跨平台無頭剪貼簿存取(文字 + 影像)+ `win32_clipboard_api.py`:**所有剪貼簿格式共用的 Win32 原型與 open/alloc/lock 流程**(`open_clipboard()` 會等過短暫被別的行程佔住的剪貼簿——Win32 一次只允許一個行程開啟,別人正在複製就必然失敗)(`argtypes` 只宣告一半曾讓四支 writer 在 64 位元上必然丟 `OverflowError`,見 CHANGELOG)。`set_clipboard_image` 同時接受 PNG 位元組與檔案路徑——先前這個名字在本子套件裡有**兩份不同簽章的實作**(`clipboard.py` 吃 bytes、`clipboard_image.py` 吃路徑),匯錯來源只會在執行期才炸,已合併成一支 | +| `utils/clipboard/` | 496 | 跨平台無頭剪貼簿存取(文字 + 影像)+ `win32_clipboard_api.py`:**所有剪貼簿格式共用的 Win32 原型與 open/alloc/lock 流程**(`open_clipboard()` 會等過短暫被別的行程佔住的剪貼簿——Win32 一次只允許一個行程開啟,別人正在複製就必然失敗)(`argtypes` 只宣告一半曾讓四支 writer 在 64 位元上必然丟 `OverflowError`,見 CHANGELOG)。`set_clipboard_image` 同時接受 PNG 位元組與檔案路徑——先前這個名字在本子套件裡有**兩份不同簽章的實作**(`clipboard.py` 吃 bytes、`clipboard_image.py` 吃路徑),匯錯來源只會在執行期才炸,已合併成一支 | | `utils/clipboard_files/` | 96 | 剪貼簿檔案清單(CF_HDROP):純 DROPFILES 封裝 + Win32 存取 | | `utils/clipboard_formats/` | 151 | 檢視與分類剪貼簿可用格式(純分類/差異 + Win32 列舉) | | `utils/clipboard_history/` | 109 | 剪貼簿歷史:環形緩衝 + 背景輪詢器 | @@ -777,7 +777,7 @@ socket server 有 8 MiB 讀取上限與 30 秒 handler timeout。 | `permissions.py` / `clipboard_sync.py` / `wake_on_lan.py` / `session_actions.py` / `auth.py` | 65 / 73 / 57 / 41 / 29 | 逐 session 權限、剪貼簿同步、WOL、SAS 注入與螢幕遮蔽、HMAC 挑戰回應。 | | `ws_host.py` / `ws_viewer.py` / `jpeg_recorder.py` | 41 / 30 / 139 | WebSocket 傳輸變體與 TCP 路徑錄影。 | -#### `utils/usb/`(4,253 行)與 `utils/usbip/`(920 行) +#### `utils/usb/`(4,281 行)與 `utils/usbip/`(920 行) | 檔案 | 行數 | 職責 | | --- | ---: | --- | @@ -1028,7 +1028,7 @@ socket 預設綁 `127.0.0.1`;資源一律用 `with`。 | `utils/mcp_server/` | 21 | 17,330 | | `utils/remote_desktop/` | 56 | 11,990 | | `utils/executor/` | 6 | 9,075 | -| `utils/usb/` | 17 | 4,253 | +| `utils/usb/` | 17 | 4,281 | | `je_auto_control/`(頂層 3 檔) | 3 | 2,363 | | `utils/accessibility/` | 13 | 2,835 | | `wrapper/` | 19 | 3,293 | @@ -1044,6 +1044,6 @@ socket 預設綁 `127.0.0.1`;資源一律用 `with`。 | `osx/` | 17 | 915 | | `autocontrol-lsp/` | 8 | 744 | | `utils/hotkey/` | 7 | 727 | -| 其餘模組(約 286 個 `utils/` 子套件 + `android/`/`ios/`/周邊小工具) | 673 | 47,498 | -| **總計** | **1,025** | **140,625** | +| 其餘模組(約 286 個 `utils/` 子套件 + `android/`/`ios/`/周邊小工具) | 673 | 47,505 | +| **總計** | **1,025** | **140,660** | diff --git a/je_auto_control/utils/clipboard/clipboard.py b/je_auto_control/utils/clipboard/clipboard.py index 3759f5e5..b188f09a 100644 --- a/je_auto_control/utils/clipboard/clipboard.py +++ b/je_auto_control/utils/clipboard/clipboard.py @@ -101,8 +101,8 @@ def _win_get() -> str: from je_auto_control.utils.clipboard.win32_clipboard_api import open_clipboard - user32 = ctypes.WinDLL("user32", use_last_error=True) - kernel32 = ctypes.WinDLL("kernel32", use_last_error=True) + user32 = ctypes.WinDLL("user32", use_last_error=True) # type: ignore[attr-defined] # reason: win32-only ctypes + kernel32 = ctypes.WinDLL("kernel32", use_last_error=True) # type: ignore[attr-defined] # reason: win32-only ctypes cf_unicodetext = 13 user32.OpenClipboard.argtypes = [wintypes.HWND] @@ -133,8 +133,8 @@ def _win_set(text: str) -> None: from je_auto_control.utils.clipboard.win32_clipboard_api import open_clipboard - user32 = ctypes.WinDLL("user32", use_last_error=True) - kernel32 = ctypes.WinDLL("kernel32", use_last_error=True) + user32 = ctypes.WinDLL("user32", use_last_error=True) # type: ignore[attr-defined] # reason: win32-only ctypes + kernel32 = ctypes.WinDLL("kernel32", use_last_error=True) # type: ignore[attr-defined] # reason: win32-only ctypes cf_unicodetext = 13 gmem_moveable = 0x0002 @@ -245,7 +245,9 @@ def _win_set_image(png_bytes: bytes) -> None: raise RuntimeError( "Pillow is required for clipboard image support" ) from error - image = Image.open(BytesIO(png_bytes)) + # `Image.open` returns an `ImageFile`; `convert` returns a plain + # `Image`, so the variable has to be declared as the wider one. + image: Image.Image = Image.open(BytesIO(png_bytes)) if image.mode != "RGB": image = image.convert("RGB") bmp_buf = BytesIO() @@ -260,8 +262,8 @@ def _win_set_image(png_bytes: bytes) -> None: open_clipboard, ) - user32 = ctypes.WinDLL("user32", use_last_error=True) - kernel32 = ctypes.WinDLL("kernel32", use_last_error=True) + user32 = ctypes.WinDLL("user32", use_last_error=True) # type: ignore[attr-defined] # reason: win32-only ctypes + kernel32 = ctypes.WinDLL("kernel32", use_last_error=True) # type: ignore[attr-defined] # reason: win32-only ctypes cf_dib = 8 gmem_moveable = 0x0002 diff --git a/je_auto_control/utils/clipboard/win32_clipboard_api.py b/je_auto_control/utils/clipboard/win32_clipboard_api.py index 028def63..c80bd655 100644 --- a/je_auto_control/utils/clipboard/win32_clipboard_api.py +++ b/je_auto_control/utils/clipboard/win32_clipboard_api.py @@ -18,7 +18,7 @@ import time from contextlib import contextmanager from ctypes import wintypes -from typing import Iterator, Optional, Tuple +from typing import Any, Iterator, Optional, Tuple GMEM_MOVEABLE = 0x0002 _OPEN_FAILED = "OpenClipboard failed" @@ -40,11 +40,16 @@ def _require_windows() -> None: raise RuntimeError("the Win32 clipboard API is only available on Windows") -def clipboard_api() -> Tuple[object, object]: - """``(user32, kernel32)`` with every clipboard prototype declared.""" +def clipboard_api() -> Tuple[Any, Any]: + """``(user32, kernel32)`` with every clipboard prototype declared. + + ``Any``, not a DLL type: a ctypes library object resolves every symbol + through ``__getattr__``, so there is nothing narrower to promise, and + ``ctypes.WinDLL`` itself is declared on the Windows target only. + """ _require_windows() - user32 = ctypes.WinDLL("user32", use_last_error=True) - kernel32 = ctypes.WinDLL("kernel32", use_last_error=True) + user32 = ctypes.WinDLL("user32", use_last_error=True) # type: ignore[attr-defined] # reason: win32-only ctypes + kernel32 = ctypes.WinDLL("kernel32", use_last_error=True) # type: ignore[attr-defined] # reason: win32-only ctypes user32.OpenClipboard.argtypes = [wintypes.HWND] user32.OpenClipboard.restype = wintypes.BOOL @@ -75,7 +80,7 @@ def clipboard_api() -> Tuple[object, object]: @contextmanager -def open_clipboard(user32: Optional[object] = None) -> Iterator[object]: +def open_clipboard(user32: Optional[Any] = None) -> Iterator[Any]: """Own the clipboard for the block, waiting out a transiently busy one. Pass the ``user32`` handle you already prototyped to keep the declarations diff --git a/je_auto_control/utils/usb/passthrough/session.py b/je_auto_control/utils/usb/passthrough/session.py index 208a69d8..da64fd25 100644 --- a/je_auto_control/utils/usb/passthrough/session.py +++ b/je_auto_control/utils/usb/passthrough/session.py @@ -329,7 +329,7 @@ def _handle_resume(self, frame: Frame) -> Frame: with self._lock: claim_id = self._resume_index.get(token) claim = self._claims.get(claim_id) if claim_id is not None else None - if claim is None: + if claim is None or claim_id is None: return _opened_failure(frame.claim_id, "unknown or expired resume token") self._audit("usb_resume", "?", "?", None, detail=f"claim_id={claim_id}") return Frame( diff --git a/je_auto_control/utils/usb/passthrough/viewer_client.py b/je_auto_control/utils/usb/passthrough/viewer_client.py index ff4e005e..fab9e48a 100644 --- a/je_auto_control/utils/usb/passthrough/viewer_client.py +++ b/je_auto_control/utils/usb/passthrough/viewer_client.py @@ -118,7 +118,7 @@ def control_transfer(self, *, bm_request_type: int, b_request: int, w_value: int = 0, w_index: int = 0, data: bytes = b"", length: int = 0, timeout_ms: int = 1000) -> bytes: - request = { + request: Dict[str, Any] = { "bm_request_type": int(bm_request_type), "b_request": int(b_request), "w_value": int(w_value), "w_index": int(w_index), diff --git a/je_auto_control/utils/usb/passthrough/winusb_backend.py b/je_auto_control/utils/usb/passthrough/winusb_backend.py index 82e99557..cf229cac 100644 --- a/je_auto_control/utils/usb/passthrough/winusb_backend.py +++ b/je_auto_control/utils/usb/passthrough/winusb_backend.py @@ -25,7 +25,7 @@ import ctypes.wintypes as wintypes import platform import re -from typing import List, Optional +from typing import Any, List, NamedTuple, Optional from je_auto_control.utils.logging.logging_instance import autocontrol_logger from je_auto_control.utils.usb.passthrough.backend import ( @@ -114,24 +114,45 @@ def _winusb_guid() -> _GUID: # --------------------------------------------------------------------------- -_setupapi: Optional[ctypes.WinDLL] = None -_winusb: Optional[ctypes.WinDLL] = None -_kernel32: Optional[ctypes.WinDLL] = None +class _Dlls(NamedTuple): + """The three loaded DLL handles, with their prototypes already bound.""" + setupapi: Any + winusb: Any + kernel32: Any -def _load_dlls() -> None: - global _setupapi, _winusb, _kernel32 - if _setupapi is not None: - return - _setupapi = ctypes.WinDLL("setupapi", use_last_error=True) - _winusb = ctypes.WinDLL("winusb", use_last_error=True) - _kernel32 = ctypes.WinDLL("kernel32", use_last_error=True) - _bind_setupapi(_setupapi) - _bind_winusb(_winusb) - _bind_kernel32(_kernel32) +_loaded: Optional[_Dlls] = None -def _bind_setupapi(dll: ctypes.WinDLL) -> None: + +def _load_dlls() -> _Dlls: + """Load and bind the three DLLs once; return them. + + The globals are published only after all three have loaded. Assigning + them one at a time meant a failure on the second call left the first + set, so the `is not None` guard short-circuited every later attempt and + the callers got `AttributeError: 'NoneType'` instead of a retry. + """ + global _loaded + if _loaded is not None: + return _loaded + setupapi = ctypes.WinDLL( # type: ignore[attr-defined] # reason: win32-only ctypes + "setupapi", use_last_error=True, + ) + winusb = ctypes.WinDLL( # type: ignore[attr-defined] # reason: win32-only ctypes + "winusb", use_last_error=True, + ) + kernel32 = ctypes.WinDLL( # type: ignore[attr-defined] # reason: win32-only ctypes + "kernel32", use_last_error=True, + ) + _bind_setupapi(setupapi) + _bind_winusb(winusb) + _bind_kernel32(kernel32) + _loaded = _Dlls(setupapi, winusb, kernel32) + return _loaded + + +def _bind_setupapi(dll: Any) -> None: dll.SetupDiGetClassDevsW.argtypes = [ ctypes.POINTER(_GUID), wintypes.LPCWSTR, wintypes.HWND, wintypes.DWORD, ] @@ -151,7 +172,7 @@ def _bind_setupapi(dll: ctypes.WinDLL) -> None: dll.SetupDiDestroyDeviceInfoList.restype = wintypes.BOOL -def _bind_winusb(dll: ctypes.WinDLL) -> None: +def _bind_winusb(dll: Any) -> None: dll.WinUsb_Initialize.argtypes = [ wintypes.HANDLE, ctypes.POINTER(wintypes.HANDLE), ] @@ -180,7 +201,7 @@ def _bind_winusb(dll: ctypes.WinDLL) -> None: dll.WinUsb_SetPipePolicy.restype = wintypes.BOOL -def _bind_kernel32(dll: ctypes.WinDLL) -> None: +def _bind_kernel32(dll: Any) -> None: dll.CreateFileW.argtypes = [ wintypes.LPCWSTR, wintypes.DWORD, wintypes.DWORD, ctypes.c_void_p, wintypes.DWORD, wintypes.DWORD, wintypes.HANDLE, @@ -212,27 +233,27 @@ def __init__(self) -> None: ) from error def list(self) -> List[BackendDevice]: + setupapi = _load_dlls().setupapi guid = _winusb_guid() - info_set = _setupapi.SetupDiGetClassDevsW( + info_set = setupapi.SetupDiGetClassDevsW( ctypes.byref(guid), None, None, _DIGCF_PRESENT | _DIGCF_DEVICEINTERFACE, ) if info_set is None or info_set == _INVALID_HANDLE_VALUE: - raise RuntimeError( - f"SetupDiGetClassDevs failed: {ctypes.get_last_error()}", - ) + last_error = ctypes.get_last_error() # type: ignore[attr-defined] # reason: win32-only ctypes + raise RuntimeError(f"SetupDiGetClassDevs failed: {last_error}") devices: List[BackendDevice] = [] try: index = 0 iface = _SP_DEVICE_INTERFACE_DATA() iface.cbSize = ctypes.sizeof(_SP_DEVICE_INTERFACE_DATA) while True: - ok = _setupapi.SetupDiEnumDeviceInterfaces( + ok = setupapi.SetupDiEnumDeviceInterfaces( info_set, None, ctypes.byref(guid), index, ctypes.byref(iface), ) if not ok: - error = ctypes.get_last_error() + error = ctypes.get_last_error() # type: ignore[attr-defined] # reason: win32-only ctypes if error == _ERROR_NO_MORE_ITEMS: break autocontrol_logger.warning( @@ -251,7 +272,7 @@ def list(self) -> List[BackendDevice]: bus_location=path, )) finally: - _setupapi.SetupDiDestroyDeviceInfoList(info_set) + setupapi.SetupDiDestroyDeviceInfoList(info_set) return devices def open(self, *, vendor_id: str, product_id: str, @@ -266,6 +287,8 @@ def open(self, *, vendor_id: str, product_id: str, for device in self.list(): if device.vendor_id != vendor_id or device.product_id != product_id: continue + if device.bus_location is None: + continue # enumerated without an interface path; unopenable return _open_handle(device.bus_location) raise RuntimeError( f"WinUSB: no device matches {vendor_id}:{product_id}", @@ -286,11 +309,12 @@ def __init__(self, file_handle: int, winusb_handle: int) -> None: def close(self) -> None: if self._closed: return + dlls = _load_dlls() try: - _winusb.WinUsb_Free(self._winusb_handle) + dlls.winusb.WinUsb_Free(self._winusb_handle) finally: try: - _kernel32.CloseHandle(self._file_handle) + dlls.kernel32.CloseHandle(self._file_handle) finally: self._closed = True @@ -314,13 +338,14 @@ def control_transfer(self, *, bm_request_type: int, b_request: int, Length=buffer_size & 0xFFFF, ) transferred = wintypes.DWORD(0) - ok = _winusb.WinUsb_ControlTransfer( + ok = _load_dlls().winusb.WinUsb_ControlTransfer( self._winusb_handle, setup, buffer, buffer_size, ctypes.byref(transferred), None, ) if not ok: + last_error = ctypes.get_last_error() # type: ignore[attr-defined] # reason: win32-only ctypes raise RuntimeError( - f"WinUsb_ControlTransfer failed: {ctypes.get_last_error()}", + f"WinUsb_ControlTransfer failed: {last_error}", ) if is_in: return bytes(buffer[: transferred.value]) @@ -353,7 +378,8 @@ def _endpoint_transfer(self, kind: str, *, endpoint: int, # Apply per-pipe timeout — WinUSB reads/writes don't take a # timeout argument directly. timeout_value = wintypes.DWORD(int(timeout_ms)) - ok = _winusb.WinUsb_SetPipePolicy( + winusb = _load_dlls().winusb + ok = winusb.WinUsb_SetPipePolicy( self._winusb_handle, endpoint & 0xFF, _PIPE_TRANSFER_TIMEOUT, ctypes.sizeof(timeout_value), ctypes.byref(timeout_value), @@ -361,30 +387,30 @@ def _endpoint_transfer(self, kind: str, *, endpoint: int, if not ok: autocontrol_logger.debug( "WinUsb_SetPipePolicy(timeout) failed: %d", - ctypes.get_last_error(), + ctypes.get_last_error(), # type: ignore[attr-defined] # reason: win32-only ctypes ) transferred = wintypes.DWORD(0) if direction == "in": buffer = (ctypes.c_ubyte * int(length))() - ok = _winusb.WinUsb_ReadPipe( + ok = winusb.WinUsb_ReadPipe( self._winusb_handle, endpoint & 0xFF, buffer, int(length), ctypes.byref(transferred), None, ) if not ok: raise RuntimeError( f"WinUsb_ReadPipe ({kind}) failed: " - f"{ctypes.get_last_error()}", + f"{ctypes.get_last_error()}", # type: ignore[attr-defined] # reason: win32-only ctypes ) return bytes(buffer[: transferred.value]) out_buffer = (ctypes.c_ubyte * len(data)).from_buffer_copy(data) - ok = _winusb.WinUsb_WritePipe( + ok = winusb.WinUsb_WritePipe( self._winusb_handle, endpoint & 0xFF, out_buffer, len(data), ctypes.byref(transferred), None, ) if not ok: raise RuntimeError( f"WinUsb_WritePipe ({kind}) failed: " - f"{ctypes.get_last_error()}", + f"{ctypes.get_last_error()}", # type: ignore[attr-defined] # reason: win32-only ctypes ) return b"" @@ -401,17 +427,18 @@ def _raise_if_closed(self) -> None: def _resolve_interface_detail(info_set: int, iface: _SP_DEVICE_INTERFACE_DATA) -> Optional[str]: """Two-call pattern: first to size the buffer, second to fill it.""" + setupapi = _load_dlls().setupapi needed = wintypes.DWORD(0) - _setupapi.SetupDiGetDeviceInterfaceDetailW( + setupapi.SetupDiGetDeviceInterfaceDetailW( info_set, ctypes.byref(iface), None, 0, ctypes.byref(needed), None, ) - if ctypes.get_last_error() != _ERROR_INSUFFICIENT_BUFFER: + if ctypes.get_last_error() != _ERROR_INSUFFICIENT_BUFFER: # type: ignore[attr-defined] # reason: win32-only ctypes return None buffer = ctypes.create_string_buffer(needed.value) # The struct begins with a DWORD cbSize — value depends on bitness. cb_size = 8 if ctypes.sizeof(ctypes.c_void_p) == 8 else 6 ctypes.memmove(buffer, ctypes.byref(wintypes.DWORD(cb_size)), 4) - ok = _setupapi.SetupDiGetDeviceInterfaceDetailW( + ok = setupapi.SetupDiGetDeviceInterfaceDetailW( info_set, ctypes.byref(iface), buffer, needed.value, None, None, ) @@ -432,7 +459,8 @@ def _parse_vid_pid(path: str) -> tuple: def _open_handle(device_path: str) -> _WinusbHandle: - file_handle = _kernel32.CreateFileW( + dlls = _load_dlls() + file_handle = dlls.kernel32.CreateFileW( device_path, _GENERIC_READ | _GENERIC_WRITE, _FILE_SHARE_READ | _FILE_SHARE_WRITE, @@ -441,13 +469,13 @@ def _open_handle(device_path: str) -> _WinusbHandle: if file_handle is None or file_handle == _INVALID_HANDLE_VALUE: raise RuntimeError( f"CreateFileW({device_path!r}) failed: " - f"{ctypes.get_last_error()}", + f"{ctypes.get_last_error()}", # type: ignore[attr-defined] # reason: win32-only ctypes ) winusb_handle = wintypes.HANDLE() - ok = _winusb.WinUsb_Initialize(file_handle, ctypes.byref(winusb_handle)) - if not ok: - last_error = ctypes.get_last_error() - _kernel32.CloseHandle(file_handle) + ok = dlls.winusb.WinUsb_Initialize(file_handle, ctypes.byref(winusb_handle)) + if not ok or winusb_handle.value is None: + last_error = ctypes.get_last_error() # type: ignore[attr-defined] # reason: win32-only ctypes + dlls.kernel32.CloseHandle(file_handle) raise RuntimeError( f"WinUsb_Initialize failed: {last_error}", ) diff --git a/test/unit_test/headless/test_usb_platform_backends.py b/test/unit_test/headless/test_usb_platform_backends.py index 4b55e97e..b7e2db51 100644 --- a/test/unit_test/headless/test_usb_platform_backends.py +++ b/test/unit_test/headless/test_usb_platform_backends.py @@ -59,11 +59,12 @@ def test_winusb_dlls_loaded(): should not re-error on import.""" from je_auto_control.utils.usb.passthrough import winusb_backend as wb WinusbBackend() - assert wb._setupapi is not None - assert wb._winusb is not None - assert wb._kernel32 is not None + assert wb._loaded is not None + dlls = wb._load_dlls() + assert dlls is wb._loaded, "the loader must be idempotent" + assert (dlls.setupapi, dlls.winusb, dlls.kernel32) != (None, None, None) # SetupDiGetClassDevsW signature was bound. - assert wb._setupapi.SetupDiGetClassDevsW.restype is not None + assert dlls.setupapi.SetupDiGetClassDevsW.restype is not None # --------------------------------------------------------------------------- diff --git a/test/verify/typing_contract_exempt.txt b/test/verify/typing_contract_exempt.txt index 74204883..4cb9563e 100644 --- a/test/verify/typing_contract_exempt.txt +++ b/test/verify/typing_contract_exempt.txt @@ -14,7 +14,7 @@ # "Does not type-check" means on at least one of the three targets mypy can be # pointed at (win32, linux, darwin) — not just the one CI happens to run on. # -# Measured entries: 92 +# Measured entries: 86 je_auto_control.__main__ je_auto_control.cli @@ -39,9 +39,6 @@ je_auto_control.utils.anchor_locator.locator je_auto_control.utils.annotate.annotate je_auto_control.utils.bulkhead.bulkhead je_auto_control.utils.change_localize.change_localize -je_auto_control.utils.clipboard.clipboard -je_auto_control.utils.clipboard.win32_clipboard_api -je_auto_control.utils.clipboard_formats.clipboard_formats je_auto_control.utils.codegen.codegen je_auto_control.utils.color_match.color_match je_auto_control.utils.color_stats.color_stats @@ -99,9 +96,6 @@ je_auto_control.utils.text_normalize.text_normalize je_auto_control.utils.text_regions.text_regions je_auto_control.utils.tls_acme.challenge je_auto_control.utils.url_canon.url_canon -je_auto_control.utils.usb.passthrough.session -je_auto_control.utils.usb.passthrough.viewer_client -je_auto_control.utils.usb.passthrough.winusb_backend je_auto_control.utils.visual_regression.compare je_auto_control.utils.watcher.watcher je_auto_control.utils.window_capture.window_capture From 31c2d5ee5c37db43fab4a927389c2149f04a6b73 Mon Sep 17 00:00:00 2001 From: JeffreyChen Date: Sat, 22 Aug 2026 20:48:32 +0800 Subject: [PATCH 13/19] Clear the GUI cluster, and three failures the mixin noise was hiding MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Thirteen of the fourteen gui modules. Most of the eighty-nine errors were the mixin shape this branch has now fixed three times: six tab mixins read `_tr`, `_translate` and `timer` off a host they never declared, and each said so in its own docstring. Those docstrings are declarations now, in a TYPE_CHECKING block the runtime strips. Three of the fixes fail for a user, not for a checker: * A pixel assertion with one coordinate reported the wrong problem. `assert_pixel(*_parse_ints(xy)[:2], _parse_ints(rgb), match=…)` binds the RGB list to `y` when the star-unpack yields one item, so `match` and `raise_on_fail` collide with positional slots and the user gets a TypeError about duplicate keyword arguments. The count is checked first now, and the message names what is missing. * `_get_mouse_pos` unpacked a value the Windows backend really returns as None — `GetCursorPos` fails on a locked or secure desktop — so the warning box read "cannot unpack non-sequence NoneType object". It raises AutoControlException with a sentence instead. * `multi_language_wrapper` typed its listeners `List[callable]`, the builtin function rather than a type, so every `listener(language)` read as calling something not callable. `recording_edit.editor` came with them: both optional parameters were written `end: int = None`. webrtc_panel.py is the one that stayed. Seven of its errors come from `_build_advanced_group(panel: TranslatableMixin, …)`, which writes five widget attributes back onto a panel that has none of them; the honest type is a Protocol, and the file sits exactly on its 2,555-line cap. Progress.md now records that its exemption is blocked on the split that file already owes — moving that builder out settles both at once. --- Progress.md | 41 +++++++++++++---- WHATS_NEW.md | 46 +++++++++++++++++++ architecture_explore.md | 6 +-- je_auto_control/gui/_auto_click_tab.py | 19 +++++++- je_auto_control/gui/_image_detect_tab.py | 8 ++++ je_auto_control/gui/_record_tab.py | 9 ++++ je_auto_control/gui/_report_tab.py | 8 ++++ je_auto_control/gui/_screenshot_tab.py | 9 ++++ je_auto_control/gui/_script_tab.py | 8 ++++ je_auto_control/gui/assertions_tab.py | 14 ++++-- .../multi_language_wrapper.py | 4 +- je_auto_control/gui/main_widget.py | 2 +- .../gui/remote_desktop/webrtc_dialogs.py | 4 +- je_auto_control/gui/test_suite_tab.py | 4 +- .../utils/recording_edit/editor.py | 8 ++-- test/verify/typing_contract_exempt.txt | 15 +----- 16 files changed, 165 insertions(+), 40 deletions(-) diff --git a/Progress.md b/Progress.md index 92b6eacd..cbe4edae 100644 --- a/Progress.md +++ b/Progress.md @@ -240,11 +240,11 @@ capability enum 值與 variadic `ei_seat_bind_capabilities`、event-type enum (`--cov-report=term-missing` 已經開著,CI 每一格都存了 `coverage.xml` artifact), 而不是齊頭式地補測試。 -### mypy:整包把關,86 個模組還沒過 +### mypy:整包把關,73 個模組還沒過 範圍不再是兩條路徑,而是**整包減去一張只准變少的清單** (`test/verify/typing_contract_exempt.txt`)。差別在於預設值:路徑清單只有人想到才會長, -新模組預設在圈外;現在新模組**預設就在契約裡**,1,017 個檔案有 931 個已經過關。 +新模組預設在圈外;現在新模組**預設就在契約裡**,1,017 個檔案有 944 個已經過關。 `wrapper` 那一群(6 個模組)在 2026-08-21 清掉了,做法見 [WHATS_NEW.md](WHATS_NEW.md): 平台縫的八個匯出名稱先宣告型別、再讓分支去綁定,其中三個用 @@ -310,12 +310,37 @@ ctypes 表面一收掉,卡在它後面的兩群也跟著清了:`utils/usb/pa 而 `object` 沒有任何屬性,所以透過它的十二個 Win32 呼叫全是型別錯誤。 做法見 [WHATS_NEW.md](WHATS_NEW.md)。 -剩下 86 個模組要清。錯誤碼分布是 `attr-defined` 佔大宗,其次 -`arg-type`/`union-attr`/`assignment`,**最大的一群現在是 `gui`** -(11+`gui/remote_desktop` 2,共 89 個錯誤),其次 `utils/mcp_server`(4,38 個)。 -以檔案計最大的三個是 `gui/_auto_click_tab.py`(31)、 -`gui/remote_desktop/webrtc_panel.py`(27)、 -`utils/mcp_server/_client_requests.py`(21)。 +`gui` 那一群(13 個模組)同日清掉,只剩 `webrtc_panel.py`(見下)。 +大宗又是 mixin 借用宿主的成員(六個分頁 mixin 借 `_tr`/`_translate`/`timer`), +做法與 webrtc 那三個 mixin 相同。順帶修掉三個真的會壞的地方: +`assertions_tab` 的 pixel 斷言把 `*_parse_ints(...)[:2]` 星號展開,使用者只打一個 +座標時 RGB 清單會被綁到 `y`,錯誤訊息變成「關鍵字參數重複」而不是「你少打了一個 +座標」;`_auto_click_tab._get_mouse_pos` 直接解包 `get_mouse_position()`,而 +Windows 後端在鎖定桌面上真的會回 `None`;`multi_language_wrapper` 的監聽器清單 +標成 `List[callable]`(`callable` 是內建函式,不是型別)。 +做法見 [WHATS_NEW.md](WHATS_NEW.md)。 + +剩下 73 個模組要清。錯誤碼分布是 `attr-defined` 佔大宗,其次 +`arg-type`/`union-attr`/`assignment`,**最大的一群現在是 `utils/mcp_server`** +(4 個模組、38 個錯誤)。以檔案計最大的三個是 +`utils/mcp_server/_client_requests.py`(21)、 +`gui/remote_desktop/webrtc_panel.py`(27)、`utils/executor/action_executor.py`(9)。 + +#### `webrtc_panel.py` 的型別豁免現在卡在它欠的那次拆檔 + +`TODO` — 27 個錯誤,但清掉它需要先讓這個檔可以變長 + +二十七個錯誤裡有七個來自 `_build_advanced_group(panel: TranslatableMixin, ...)`: +這個自由函式**會把五個 widget 屬性寫回 panel 上**(`_stun_edit`、`_turn_edit`、 +`_turn_user_edit`、`_turn_cred_edit`、`_hw_codec_combo`),而 `TranslatableMixin` +一個都沒有。正確的型別是一個 Protocol,把它需要與寫入的成員寫出來——但 +`webrtc_panel.py` **正好卡在 2,555 行的上限**(見本檔開頭那張表:只准變短), +而加 Protocol 一定變長。 + +真正的解法就是那張表早就寫著的:把這個檔拆成 panel + 各控制器。 +`_build_advanced_group` 本身就是第一個該搬出去的——它是共用的 widget group builder, +不屬於 panel 模組,搬走同時解決長度與型別兩件事。**在那之前這個模組留在豁免清單上, +理由記在這裡,不是被忘記。** **下一步**是一次清一個群集,清完把行從清單刪掉—— `python test/verify/typing_contract_verify.py --fix` 會替你改,CI 會在你忘了刪的時候紅掉。 已經清掉的那些留下四個可以直接套用的樣板:mixin 用 `if TYPE_CHECKING:` 宣告 diff --git a/WHATS_NEW.md b/WHATS_NEW.md index d7488c0f..ca7132e8 100644 --- a/WHATS_NEW.md +++ b/WHATS_NEW.md @@ -208,6 +208,52 @@ comparison as a `DECIDE`, because the cleanest of them changes what the gate means rather than what the code says. **136 modules left, 881 of 1,017 files inside the contract.** +### The GUI Cluster: Three Real Failures Behind the Mixin Noise + +Thirteen of the fourteen `gui` modules came off the list. Most of the eighty-nine +errors were the mixin shape already fixed twice on this branch — six tab mixins +read `self._tr`, `self._translate` and `self.timer` off a host they never +declared, and every one of them said so in its own docstring +("Requires the host widget to expose…"). Those docstrings are now +`if TYPE_CHECKING:` declarations, stripped at runtime. + +Underneath them were three things that fail for a user, not for a checker: + +- **A pixel assertion with one coordinate reported the wrong problem.** + `assertions_tab` called + `assert_pixel(*_parse_ints(self._xy.text())[:2], _parse_ints(self._rgb.text()), …)`. + Type `5` instead of `5,6` and the star-unpack contributes one argument, so the + RGB list binds to `y`, `match=` and `raise_on_fail=` collide with the + positional slots, and the user sees a `TypeError` about duplicate keyword + arguments. The count is checked first now, and the message names what is + missing. +- **`_get_mouse_pos` unpacked a value the Windows backend really does return + as `None`.** `win32_ctype_mouse_control.position()` returns `None` when + `GetCursorPos` fails — which is what happens on a locked or secure desktop — + and the tab did `x, y = get_mouse_position()`. The existing `except TypeError` + caught it and showed "cannot unpack non-sequence NoneType object". It raises + `AutoControlException` with a sentence instead. +- **`multi_language_wrapper` typed its listener list `List[callable]`.** + `callable` is the builtin *function*, not a type, so mypy read every + `listener(language)` call as calling something that is not callable. It is + `List[Callable[[str], None]]` now. + +`recording_edit.editor` came along with them: both of its optional parameters +were written `end: int = None`, which PEP 484 prohibits and `no_implicit_optional` +rejects. + +**`webrtc_panel.py` is the one that stayed**, and its reason is now in +`Progress.md` rather than in nobody's head. Seven of its twenty-seven errors +come from `_build_advanced_group(panel: TranslatableMixin, …)`, a free function +that *writes* five widget attributes back onto the panel — none of which +`TranslatableMixin` has. The correct type is a Protocol naming what it reads and +writes, and the file is sitting exactly on its 2,555-line cap, which may only +shrink. The real fix is the split that file already owes: `_build_advanced_group` +is a shared widget-group builder that does not belong in the panel module, and +moving it out settles the length and the type in one go. + +**73 modules left, 944 of 1,017 files inside the contract.** + ### The WinUSB Backend Was One Failed DLL Load Away From Never Recovering With the ctypes surface settled, the two clusters behind it came off: diff --git a/architecture_explore.md b/architecture_explore.md index 1c55de13..8760988e 100644 --- a/architecture_explore.md +++ b/architecture_explore.md @@ -20,7 +20,7 @@ iOS(WebDriverAgent)。核心能力是滑鼠/鍵盤控制、影像辨識、 | 指標 | 數值 | | --- | ---: | | Python 模組總數(含周邊子專案) | 1,031 | -| 程式碼總行數 | 140,725 | +| 程式碼總行數 | 140,792 | | `je_auto_control/utils/` 子套件數 | 310 | | `AC_*` 動作指令數(`known_commands()` 實測) | 773 | | 套件門面 `__all__` 公開名稱數 | 1,238 | @@ -1024,7 +1024,7 @@ socket 預設綁 `127.0.0.1`;資源一律用 `with`。 | 層/子系統 | 檔案數 | 行數 | | --- | ---: | ---: | -| `gui/` | 89 | 26,542 | +| `gui/` | 89 | 26,609 | | `utils/mcp_server/` | 21 | 17,330 | | `utils/remote_desktop/` | 56 | 11,990 | | `utils/executor/` | 6 | 9,075 | @@ -1045,5 +1045,5 @@ socket 預設綁 `127.0.0.1`;資源一律用 `with`。 | `autocontrol-lsp/` | 8 | 744 | | `utils/hotkey/` | 7 | 727 | | 其餘模組(約 286 個 `utils/` 子套件 + `android/`/`ios/`/周邊小工具) | 673 | 47,505 | -| **總計** | **1,025** | **140,660** | +| **總計** | **1,025** | **140,727** | diff --git a/je_auto_control/gui/_auto_click_tab.py b/je_auto_control/gui/_auto_click_tab.py index d6f9b042..eb262b35 100644 --- a/je_auto_control/gui/_auto_click_tab.py +++ b/je_auto_control/gui/_auto_click_tab.py @@ -1,3 +1,5 @@ +from typing import TYPE_CHECKING, Any, Callable + from PySide6.QtGui import QIntValidator from PySide6.QtWidgets import ( QWidget, QLineEdit, QComboBox, QVBoxLayout, QLabel, @@ -23,6 +25,14 @@ class AutoClickTabMixin: (``self._tr(...)``) set up by its __init__. """ + if TYPE_CHECKING: + # Declared, never defined: the widget this mixin is mixed into owns + # every one of these. The block is stripped at runtime, so nothing + # here can shadow what the host actually binds. + _tr: Callable[..., Any] + _translate: Callable[[str], str] + timer: Any + def _build_auto_click_tab(self) -> QWidget: tab = QWidget() outer = QVBoxLayout() @@ -234,7 +244,14 @@ def _do_click(self): def _get_mouse_pos(self): try: - x, y = get_mouse_position() + position = get_mouse_position() + if position is None: + # GetCursorPos fails on a locked or secure desktop; the + # backend reports that as None rather than raising. + raise AutoControlException( + "the OS did not report a cursor position", + ) + x, y = position self._pos_label_suffix = f" ({x}, {y})" self.pos_label.setText( self._translate("current_position") + self._pos_label_suffix, diff --git a/je_auto_control/gui/_image_detect_tab.py b/je_auto_control/gui/_image_detect_tab.py index 983ae4e3..dbc51953 100644 --- a/je_auto_control/gui/_image_detect_tab.py +++ b/je_auto_control/gui/_image_detect_tab.py @@ -1,4 +1,6 @@ """Image-detection tab builder (extracted mixin).""" +from typing import TYPE_CHECKING, Any, Callable + from PySide6.QtGui import QDoubleValidator from PySide6.QtWidgets import ( QCheckBox, QFileDialog, QGridLayout, QLabel, QLineEdit, QMessageBox, @@ -26,6 +28,12 @@ class ImageDetectTabMixin: left button otherwise. """ + if TYPE_CHECKING: + # Declared, never defined: the widget this mixin is mixed into owns + # every one of these. The block is stripped at runtime, so nothing + # here can shadow what the host actually binds. + _tr: Callable[..., Any] + def _build_image_detect_tab(self) -> QWidget: tab = QWidget() layout = QVBoxLayout() diff --git a/je_auto_control/gui/_record_tab.py b/je_auto_control/gui/_record_tab.py index 1e468c9e..558c73e3 100644 --- a/je_auto_control/gui/_record_tab.py +++ b/je_auto_control/gui/_record_tab.py @@ -1,6 +1,8 @@ """Record / playback tab builder (extracted mixin).""" import json +from typing import TYPE_CHECKING, Any, Callable + from PySide6.QtWidgets import ( QFileDialog, QLabel, QMessageBox, QTextEdit, QVBoxLayout, QWidget, ) @@ -27,6 +29,13 @@ class RecordTabMixin: last recording. """ + if TYPE_CHECKING: + # Declared, never defined: the widget this mixin is mixed into owns + # every one of these. The block is stripped at runtime, so nothing + # here can shadow what the host actually binds. + _tr: Callable[..., Any] + _translate: Callable[[str], str] + def _build_record_tab(self) -> QWidget: tab = QWidget() layout = QVBoxLayout() diff --git a/je_auto_control/gui/_report_tab.py b/je_auto_control/gui/_report_tab.py index 5c2318ce..a0ff5294 100644 --- a/je_auto_control/gui/_report_tab.py +++ b/je_auto_control/gui/_report_tab.py @@ -1,4 +1,6 @@ """Report-generation tab builder (extracted mixin).""" +from typing import TYPE_CHECKING, Any, Callable + from PySide6.QtWidgets import ( QGroupBox, QHBoxLayout, QLabel, QLineEdit, QTextEdit, QVBoxLayout, QWidget, @@ -18,6 +20,12 @@ class ReportTabMixin: so every label/button registers for live language switching. """ + if TYPE_CHECKING: + # Declared, never defined: the widget this mixin is mixed into owns + # every one of these. The block is stripped at runtime, so nothing + # here can shadow what the host actually binds. + _tr: Callable[..., Any] + def _build_report_tab(self) -> QWidget: tab = QWidget() layout = QVBoxLayout() diff --git a/je_auto_control/gui/_screenshot_tab.py b/je_auto_control/gui/_screenshot_tab.py index a86c73d0..67c3243a 100644 --- a/je_auto_control/gui/_screenshot_tab.py +++ b/je_auto_control/gui/_screenshot_tab.py @@ -1,4 +1,6 @@ """Screenshot / pixel-probe tab builder (extracted mixin).""" +from typing import TYPE_CHECKING, Any, Callable + from PySide6.QtGui import QIntValidator from PySide6.QtWidgets import ( QFileDialog, QGridLayout, QGroupBox, QHBoxLayout, QLabel, QLineEdit, @@ -23,6 +25,13 @@ class ScreenshotTabMixin: switching. """ + if TYPE_CHECKING: + # Declared, never defined: the widget this mixin is mixed into owns + # every one of these. The block is stripped at runtime, so nothing + # here can shadow what the host actually binds. + _tr: Callable[..., Any] + _translate: Callable[[str], str] + def _build_screenshot_tab(self) -> QWidget: tab = QWidget() layout = QVBoxLayout() diff --git a/je_auto_control/gui/_script_tab.py b/je_auto_control/gui/_script_tab.py index d9d7fa10..ec2a89fa 100644 --- a/je_auto_control/gui/_script_tab.py +++ b/je_auto_control/gui/_script_tab.py @@ -1,6 +1,8 @@ """Script-executor tab builder (extracted mixin).""" import json +from typing import TYPE_CHECKING, Any, Callable + from PySide6.QtWidgets import ( QFileDialog, QHBoxLayout, QLabel, QLineEdit, QTextEdit, QVBoxLayout, QWidget, @@ -26,6 +28,12 @@ class ScriptTabMixin: Host widget must expose the ``TranslatableMixin`` API (``self._tr(...)``). """ + if TYPE_CHECKING: + # Declared, never defined: the widget this mixin is mixed into owns + # every one of these. The block is stripped at runtime, so nothing + # here can shadow what the host actually binds. + _tr: Callable[..., Any] + def _build_script_tab(self) -> QWidget: tab = QWidget() layout = QVBoxLayout() diff --git a/je_auto_control/gui/assertions_tab.py b/je_auto_control/gui/assertions_tab.py index bbc558f2..9c9a2120 100644 --- a/je_auto_control/gui/assertions_tab.py +++ b/je_auto_control/gui/assertions_tab.py @@ -4,7 +4,7 @@ Assertions run with ``raise_on_fail=False`` so the GUI reports the outcome instead of crashing the tab. """ -from typing import Any, Dict, Optional +from typing import Any, Dict, List, Optional from PySide6.QtWidgets import ( QCheckBox, QComboBox, QHBoxLayout, QLabel, QLineEdit, @@ -24,7 +24,7 @@ def _t(key: str) -> str: return language_wrapper.translate(key, key) -def _parse_ints(raw: str): +def _parse_ints(raw: str) -> List[int]: return [int(part.strip()) for part in raw.split(",") if part.strip()] @@ -107,8 +107,16 @@ def _run_assertion(self) -> Dict[str, Any]: self._target.text(), present=present, raise_on_fail=False, ).to_dict() if kind == "pixel": + coords = _parse_ints(self._xy.text()) + if len(coords) < 2: + # Unpacking a short list used to reach assert_pixel with the + # RGB list bound to `y`, and the user saw a TypeError about + # keyword arguments instead of what they typed wrong. + raise ValueError( + f"pixel assertion needs 'x,y'; got {self._xy.text()!r}", + ) return ac.assert_pixel( - *_parse_ints(self._xy.text())[:2], _parse_ints(self._rgb.text()), + coords[0], coords[1], _parse_ints(self._rgb.text()), match=present, raise_on_fail=False, ).to_dict() if kind == "window": diff --git a/je_auto_control/gui/language_wrapper/multi_language_wrapper.py b/je_auto_control/gui/language_wrapper/multi_language_wrapper.py index 29fdf17b..6b1295f3 100644 --- a/je_auto_control/gui/language_wrapper/multi_language_wrapper.py +++ b/je_auto_control/gui/language_wrapper/multi_language_wrapper.py @@ -4,7 +4,7 @@ dict of key → translation. Missing keys fall through to the English default so new features degrade gracefully before their translations land. """ -from typing import Dict, List +from typing import Callable, Dict, List from je_auto_control.gui.language_wrapper.english import english_word_dict from je_auto_control.gui.language_wrapper.japanese import japanese_word_dict @@ -29,7 +29,7 @@ def __init__(self) -> None: "Japanese": japanese_word_dict, } self.language: str = "English" - self._listeners: List[callable] = [] + self._listeners: List[Callable[[str], None]] = [] self.language_word_dict: dict = self._merged(self.language) @property diff --git a/je_auto_control/gui/main_widget.py b/je_auto_control/gui/main_widget.py index b84a7e25..f6edc87e 100644 --- a/je_auto_control/gui/main_widget.py +++ b/je_auto_control/gui/main_widget.py @@ -52,7 +52,7 @@ from je_auto_control.gui.remote_desktop_tab import RemoteDesktopTab _REMOTE_DESKTOP_IMPORT_ERROR: Optional[ImportError] = None except ImportError as _remote_desktop_error: - RemoteDesktopTab = None # type: ignore[assignment] + RemoteDesktopTab = None # type: ignore[assignment,misc] # reason: name is a class or None _REMOTE_DESKTOP_IMPORT_ERROR = _remote_desktop_error from je_auto_control.gui.rest_api_tab import RestApiTab from je_auto_control.gui.run_history_tab import RunHistoryTab diff --git a/je_auto_control/gui/remote_desktop/webrtc_dialogs.py b/je_auto_control/gui/remote_desktop/webrtc_dialogs.py index 9905a843..dfe7381c 100644 --- a/je_auto_control/gui/remote_desktop/webrtc_dialogs.py +++ b/je_auto_control/gui/remote_desktop/webrtc_dialogs.py @@ -7,7 +7,7 @@ """ from __future__ import annotations -from typing import Optional +from typing import Any, Optional from PySide6.QtCore import Qt, Signal from PySide6.QtWidgets import ( @@ -426,7 +426,7 @@ def __init__(self, parent: Optional[QWidget] = None) -> None: layout.addLayout(button_row) # Defer browser start until the dialog is shown so we don't burn # mDNS sockets when the dialog is constructed lazily. - self._browser = None + self._browser: Optional[Any] = None self._start_browser() def _start_browser(self) -> None: diff --git a/je_auto_control/gui/test_suite_tab.py b/je_auto_control/gui/test_suite_tab.py index 53f23f9b..319196c2 100644 --- a/je_auto_control/gui/test_suite_tab.py +++ b/je_auto_control/gui/test_suite_tab.py @@ -4,7 +4,7 @@ report writers, and the quarantine store. Holds no business logic. """ import json -from typing import Optional +from typing import Any, Optional from PySide6.QtCore import Qt from PySide6.QtWidgets import ( @@ -46,7 +46,7 @@ def __init__(self, parent: Optional[QWidget] = None) -> None: self._table.horizontalHeader().setStretchLastSection(True) self._summary = QLabel() self._quarantine = QListWidget() - self._last_result = None + self._last_result: Optional[Any] = None self._apply_headers() self._build_layout() self._refresh_quarantine() diff --git a/je_auto_control/utils/recording_edit/editor.py b/je_auto_control/utils/recording_edit/editor.py index cfd2646d..dfcfa091 100644 --- a/je_auto_control/utils/recording_edit/editor.py +++ b/je_auto_control/utils/recording_edit/editor.py @@ -3,11 +3,11 @@ All functions return new lists rather than mutating the input so callers can preserve the original recording. """ -from typing import Callable, List +from typing import Callable, List, Optional -def trim_actions(actions: List[list], start: int = 0, end: int = None - ) -> List[list]: +def trim_actions(actions: List[list], start: int = 0, + end: Optional[int] = None) -> List[list]: """Return a slice ``actions[start:end]`` (end=None means to the end).""" return list(actions[start:end]) @@ -84,7 +84,7 @@ def _action_name(action: object) -> object: def dedupe_moves(actions: List[list], - move_commands: List[str] = None) -> List[list]: + move_commands: Optional[List[str]] = None) -> List[list]: """Collapse each run of consecutive mouse-move actions into its last one. A raw recording captures one action per cursor sample, so a single diff --git a/test/verify/typing_contract_exempt.txt b/test/verify/typing_contract_exempt.txt index 4cb9563e..55816b40 100644 --- a/test/verify/typing_contract_exempt.txt +++ b/test/verify/typing_contract_exempt.txt @@ -14,23 +14,11 @@ # "Does not type-check" means on at least one of the three targets mypy can be # pointed at (win32, linux, darwin) — not just the one CI happens to run on. # -# Measured entries: 86 +# Measured entries: 73 je_auto_control.__main__ je_auto_control.cli -je_auto_control.gui._auto_click_tab -je_auto_control.gui._image_detect_tab -je_auto_control.gui._record_tab -je_auto_control.gui._report_tab -je_auto_control.gui._screenshot_tab -je_auto_control.gui._script_tab -je_auto_control.gui.assertions_tab -je_auto_control.gui.language_wrapper.multi_language_wrapper -je_auto_control.gui.main_widget -je_auto_control.gui.recording_editor_tab -je_auto_control.gui.remote_desktop.webrtc_dialogs je_auto_control.gui.remote_desktop.webrtc_panel -je_auto_control.gui.test_suite_tab je_auto_control.utils.act_modes.act_modes je_auto_control.utils.actionability.actionability je_auto_control.utils.admin.admin_client @@ -78,7 +66,6 @@ je_auto_control.utils.profiler.resource_profiler je_auto_control.utils.project.create_project_structure je_auto_control.utils.qr.qr je_auto_control.utils.rate_limit.rate_limit -je_auto_control.utils.recording_edit.editor je_auto_control.utils.redaction.policies je_auto_control.utils.redaction.rules je_auto_control.utils.rotated_match.rotated_match From f322a53d7d07d56b77ee484c0fe0082a8e069281 Mon Sep 17 00:00:00 2001 From: JeffreyChen Date: Sat, 22 Aug 2026 21:01:57 +0800 Subject: [PATCH 14/19] Fix a command that had never worked, and clear the MCP cluster MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit `AC_normalize_url` and `ac_normalize_url` both forward to `url_canon.normalize_url` and both passed `drop_fragment=` — the name they expose to callers. The function's parameter is `strip_fragment`, so every call raised TypeError, in the executor, in the MCP tool, and from the Script Builder field that feeds them. The outward name stays (it is in the action schema and the tool registry); the two call sites pass it under the name the callee uses. Measured: the MCP tool now returns {"url": "https://example.com/b"} where it returned an error. The rest of the cluster was shapes this branch keeps meeting: * ClientRequestMixin borrowed seven attributes from MCPServer and listed all seven in its docstring; that list is a declaration now. * `_DISPATCH_ERRORS` and `_TOOL_INVOKE_ERRORS` — the containment boundary for the whole stdio loop — were not typed as tuples of exception classes, so all three `except` sites were errors. * `_dispatch` fed an Optional[str] method name to `dict.get`. A request with no method takes the not-found branch explicitly now, with the same -32601 body it produced by falling through. * The subscription callback was a default-argument lambda, which mypy cannot infer against a `Callable[[], None]`. `functools.partial` binds the same value and states the type. Two public return annotations were wrong in the safe direction: `set_mouse_position` and `hotkey` say `... | None` while every path returns the tuple or raises. Narrowing them is what let the MCP handlers stop indexing an Optional. `get_mouse_position` keeps its `| None` — the Windows backend really does return that when GetCursorPos fails. --- Progress.md | 19 ++++++---- WHATS_NEW.md | 36 +++++++++++++++++++ architecture_explore.md | 30 ++++++++-------- .../utils/executor/action_executor.py | 4 ++- je_auto_control/utils/mcp_server/__main__.py | 11 +++--- .../utils/mcp_server/_client_requests.py | 21 +++++++++-- je_auto_control/utils/mcp_server/_protocol.py | 8 +++-- je_auto_control/utils/mcp_server/server.py | 6 +++- .../utils/mcp_server/tools/_handlers.py | 10 +++--- .../utils/webrunner_bridge/bridge.py | 4 +-- .../wrapper/auto_control_keyboard.py | 2 +- je_auto_control/wrapper/auto_control_mouse.py | 2 +- test/verify/typing_contract_exempt.txt | 6 +--- 13 files changed, 111 insertions(+), 48 deletions(-) diff --git a/Progress.md b/Progress.md index cbe4edae..514137b3 100644 --- a/Progress.md +++ b/Progress.md @@ -240,11 +240,11 @@ capability enum 值與 variadic `ei_seat_bind_capabilities`、event-type enum (`--cov-report=term-missing` 已經開著,CI 每一格都存了 `coverage.xml` artifact), 而不是齊頭式地補測試。 -### mypy:整包把關,73 個模組還沒過 +### mypy:整包把關,69 個模組還沒過 範圍不再是兩條路徑,而是**整包減去一張只准變少的清單** (`test/verify/typing_contract_exempt.txt`)。差別在於預設值:路徑清單只有人想到才會長, -新模組預設在圈外;現在新模組**預設就在契約裡**,1,017 個檔案有 944 個已經過關。 +新模組預設在圈外;現在新模組**預設就在契約裡**,1,017 個檔案有 948 個已經過關。 `wrapper` 那一群(6 個模組)在 2026-08-21 清掉了,做法見 [WHATS_NEW.md](WHATS_NEW.md): 平台縫的八個匯出名稱先宣告型別、再讓分支去綁定,其中三個用 @@ -320,11 +320,16 @@ Windows 後端在鎖定桌面上真的會回 `None`;`multi_language_wrapper` 標成 `List[callable]`(`callable` 是內建函式,不是型別)。 做法見 [WHATS_NEW.md](WHATS_NEW.md)。 -剩下 73 個模組要清。錯誤碼分布是 `attr-defined` 佔大宗,其次 -`arg-type`/`union-attr`/`assignment`,**最大的一群現在是 `utils/mcp_server`** -(4 個模組、38 個錯誤)。以檔案計最大的三個是 -`utils/mcp_server/_client_requests.py`(21)、 -`gui/remote_desktop/webrtc_panel.py`(27)、`utils/executor/action_executor.py`(9)。 +`utils/mcp_server` 那一群(4 個模組)同日清掉,順帶抓到一個**從來沒成功過的指令**: +`AC_normalize_url`/`ac_normalize_url` 都把 `drop_fragment=` 傳給 +`url_canon.normalize_url`,而那支函式的參數叫 `strip_fragment`——每一次呼叫都是 +`TypeError`。對外的名字不動(動作 schema 與工具註冊表都用它),改的是兩個呼叫點。 +做法見 [WHATS_NEW.md](WHATS_NEW.md)。 + +剩下 69 個模組要清,而且**不再有大群集**:最大的檔案是 +`gui/remote_desktop/webrtc_panel.py`(27,見下),其餘每個檔都在 8 個錯誤以下 +(`utils/executor/action_executor.py` 8,`utils/table_grid_fill`/`utils/url_canon`/ +`utils/window_zorder`/`utils/plugin_loader` 各 6)。 #### `webrtc_panel.py` 的型別豁免現在卡在它欠的那次拆檔 diff --git a/WHATS_NEW.md b/WHATS_NEW.md index ca7132e8..cfc3abec 100644 --- a/WHATS_NEW.md +++ b/WHATS_NEW.md @@ -208,6 +208,42 @@ comparison as a `DECIDE`, because the cleanest of them changes what the gate means rather than what the code says. **136 modules left, 881 of 1,017 files inside the contract.** +### `normalize_url` Had Never Worked, on Either Surface + +The MCP cluster came off next, and the gate found a command that could not +succeed. `AC_normalize_url` and `ac_normalize_url` both forward to +`url_canon.normalize_url`, and both passed `drop_fragment=` — the name they +expose to callers. The function's parameter is `strip_fragment`. Every call, +with or without that flag, raised +`TypeError: normalize_url() got an unexpected keyword argument 'drop_fragment'`: +in the executor, in the MCP tool, and from the Script Builder field that feeds +them. The outward name is unchanged (it is in the action schema and the tool +registry); the two call sites now pass it through under the name the callee +uses. Measured before and after: the MCP tool returns +`{"url": "https://example.com/b"}` where it used to return an error. + +The rest of the cluster was the shapes this branch keeps meeting: + +- **`ClientRequestMixin` borrowed seven attributes from `MCPServer`** and listed + all seven in its docstring; that list is a `TYPE_CHECKING` declaration now. +- **Two catch tuples again.** `_DISPATCH_ERRORS` and `_TOOL_INVOKE_ERRORS` are + the containment boundary for the whole stdio loop, and neither was typed as a + tuple of exception classes, so all three `except` sites were errors. +- **`_dispatch` fed an `Optional[str]` method name to `dict.get`.** A JSON-RPC + request with no `method` now takes the not-found branch explicitly, with the + same `-32601` response body it produced by falling through. +- **The subscription callback was a default-argument lambda** + (`lambda u=uri: …`), which mypy cannot infer against a `Callable[[], None]` + parameter. `functools.partial` binds `uri` the same way and says the type. + +Two public return annotations were also wrong in the safe direction: +`set_mouse_position` and `hotkey` are declared `... | None` but every path +either returns the tuple or raises. Narrowing them is what let the MCP handlers +stop indexing an Optional. `get_mouse_position` keeps its `| None` — the Windows +backend really does return that. + +**69 modules left, 948 of 1,017 files inside the contract.** + ### The GUI Cluster: Three Real Failures Behind the Mixin Noise Thirteen of the fourteen `gui` modules came off the list. Most of the eighty-nine diff --git a/architecture_explore.md b/architecture_explore.md index 8760988e..d16a3560 100644 --- a/architecture_explore.md +++ b/architecture_explore.md @@ -20,7 +20,7 @@ iOS(WebDriverAgent)。核心能力是滑鼠/鍵盤控制、影像辨識、 | 指標 | 數值 | | --- | ---: | | Python 模組總數(含周邊子專案) | 1,031 | -| 程式碼總行數 | 140,792 | +| 程式碼總行數 | 140,818 | | `je_auto_control/utils/` 子套件數 | 310 | | `AC_*` 動作指令數(`known_commands()` 實測) | 773 | | 套件門面 `__all__` 公開名稱數 | 1,238 | @@ -268,7 +268,7 @@ socket server 有 8 MiB 讀取上限與 30 秒 handler timeout。 ### 5.4.1 執行引擎與腳本資產 -> 24 個套件、約 12,900 行。 +> 24 個套件、約 12,902 行。 | 模組 | 行數 | 職責 | | --- | ---: | --- | @@ -279,7 +279,7 @@ socket server 有 8 MiB 讀取上限與 30 秒 handler timeout。 | `utils/dag/` | 475 | 跨主機 DAG 編排器(圖模型 + runner) | | `utils/decision_table/` | 103 | DMN 風格決策表:規則 + 命中策略,把分支外部化 | | `utils/deterministic/` | 96 | 決定性執行控制:固定亂數種子 + 凍結時鐘 | -| `utils/executor/` | 9,075 | **核心**。`Executor` 指令分派表(773 個 `AC_*`)、參數插值、乾跑、逐步 callback;`flow_control` 提供 34 個區塊指令(迴圈/分支/try/巨集/變數) | +| `utils/executor/` | 9,077 | **核心**。`Executor` 指令分派表(773 個 `AC_*`)、參數插值、乾跑、逐步 callback;`flow_control` 提供 34 個區塊指令(迴圈/分支/try/巨集/變數) | | `utils/flow_debugger/` | 136 | action list 的單步除錯器與追蹤器 | | `utils/input_macro/` | 342 | 定時輸入事件:錄製結果的整形(`timeline`/`InputRecorder`,Windows 與 macOS 共用)、重播與宣告式輸入序列 DSL | | `utils/json/` | 74 | action JSON 檔讀寫與正規化格式化(`fmt --check` 的後端) | @@ -490,7 +490,7 @@ socket server 有 8 MiB 讀取上限與 30 秒 handler timeout。 ### 5.4.9 AI / Agent / LLM -> 13 個套件、約 20,617 行。 +> 13 個套件、約 20,641 行。 | 模組 | 行數 | 職責 | | --- | ---: | --- | @@ -503,7 +503,7 @@ socket server 有 8 MiB 讀取上限與 30 秒 handler timeout。 | `utils/cua_action/` | 127 | 標準化 computer-use 動作結構(Anthropic/OpenAI → `AC_*`) | | `utils/llm/` | 357 | 自然語言 → action list 規劃器 + Anthropic/null 後端 | | `utils/mcp_registry/` | 92 | MCP registry `server.json` 資訊清單產生(可被發現) | -| `utils/mcp_server/` | 17,330 | **無頭 MCP 伺服器**(16K LOC,預設註冊 676 個工具=657 個 `ac_*` + 19 個別名):stdio + HTTP 傳輸、工具工廠與處理器、資源、prompt、稽核、限流、外掛熱重載 | +| `utils/mcp_server/` | 17,354 | **無頭 MCP 伺服器**(16K LOC,預設註冊 676 個工具=657 個 `ac_*` + 19 個別名):stdio + HTTP 傳輸、工具工廠與處理器、資源、prompt、稽核、限流、外掛熱重載 | | `utils/tool_use_schema/` | 180 | 把 `AC_*` 指令匯出成 Claude/OpenAI 的 tool-use schema | | `utils/trajectory_eval/` | 106 | agent 軌跡評估:依評分規準為一次執行打分 | | `utils/vision/` | 448 | VLM 元素定位器(依描述找元素)+ Anthropic/OpenAI/null 後端 | @@ -692,27 +692,27 @@ socket server 有 8 MiB 讀取上限與 30 秒 handler timeout。 上表以子套件為單位;以下把行數最大的幾個子系統展開到檔案層。 -#### `utils/executor/`(9,075 行)— 執行核心 +#### `utils/executor/`(9,077 行)— 執行核心 | 檔案 | 行數 | 職責 | | --- | ---: | --- | -| `action_executor.py` | 8,125 | `Executor` 類別與 `event_dict` 分派表(773 個指令),另含數百個把 utils 能力接成指令的 adapter 函式;全域單例 `executor` 與 `add_command_to_executor()` 擴充點。 | +| `action_executor.py` | 8,127 | `Executor` 類別與 `event_dict` 分派表(773 個指令),另含數百個把 utils 能力接成指令的 adapter 函式;全域單例 `executor` 與 `add_command_to_executor()` 擴充點。 | | `flow_control.py` | 530 | 真正的流程控制:`AC_loop`/`AC_for_each`/`AC_while_*`/`AC_if_*`/`AC_try`/`AC_retry`/`AC_parallel`/`AC_define_macro`/`AC_call_macro`/變數指令(`AC_set_var`/`AC_get_var`/`AC_inc_var`)。`LoopBreak`/`LoopContinue` 以例外實作。34 個區塊指令的分派表 `BLOCK_COMMANDS` 也在這裡,含下一列匯入的資料來源指令。 | | `flow_data_commands.py` | 253 | `AC_*_to_var` 資料來源與轉換指令:shell、時鐘、亂數、PDF、TOTP、SQL、檔案、HTTP、OCR,加上 `AC_assert_var`/`AC_assert_db`/`AC_assert_duration`/`AC_transform_var`。都不執行巢狀 action list,所以沒有迴圈/分支語意。 | | `action_schema.py` | 128 | action list 的結構驗證:形狀、參數型別、未知指令拒絕。單一走訪同時支援兩種消費方式:`validate_actions()` 遇到第一個問題就拋、`unknown_command_names()` 收齊全部不認得的名字(REST `/execute` 用它回 400)。 | | `mouse_aliases.py` | 39 | 單鍵點擊別名(`AC_click_left` 等),executor 與 callback executor 共用。 | -#### `utils/mcp_server/`(17,330 行,676 個工具)— 最大子系統 +#### `utils/mcp_server/`(17,354 行,676 個工具)— 最大子系統 | 檔案 | 行數 | 職責 | | --- | ---: | --- | | `tools/_factories.py` | 8,739 | 工具工廠:每個函式回傳一個領域的 `MCPTool` 清單(把 `AC_*` 能力包成 MCP 工具)。 | | `tools/_handlers.py` | 4,651 | 把 MCP 工具呼叫橋接到 AutoControl 無頭 API 的 adapter。 | -| `server.py` | 713 | JSON-RPC 2.0 over stdio 的最小 MCP 伺服器:連線範圍狀態、行內/併發分派、工具與 resource/prompt 處理器。 | +| `server.py` | 717 | JSON-RPC 2.0 over stdio 的最小 MCP 伺服器:連線範圍狀態、行內/併發分派、工具與 resource/prompt 處理器。 | | `http_transport.py` | 521 | MCP 的 HTTP 傳輸。 | | `http_sessions.py` | 234 | MCP 的 HTTP 傳輸用的 session 身分:`Mcp-Session-Id` 註冊表,以及每個 session 那條常駐的 server→client SSE 串流。 | -| `_client_requests.py` | 217 | 伺服器主動送出的請求:`roots/list`/`elicitation/create`/`sampling/createMessage`,對應表與回應路由,以及破壞性工具的確認交握。 | -| `_protocol.py` | 165 | JSON-RPC 線路格式:版本與識別常數、`_MCPError`、決定失敗工具行為的錯誤 tuple、envelope 產生器、工具回傳值轉 `content` 區塊。不碰伺服器狀態。 | +| `_client_requests.py` | 232 | 伺服器主動送出的請求:`roots/list`/`elicitation/create`/`sampling/createMessage`,對應表與回應路由,以及破壞性工具的確認交握。 | +| `_protocol.py` | 167 | JSON-RPC 線路格式:版本與識別常數、`_MCPError`、決定失敗工具行為的錯誤 tuple、envelope 產生器、工具回傳值轉 `content` 區塊。不碰伺服器狀態。 | | `resources.py` | 303 | MCP resource 提供者。 | | `prompts.py` | 220 | MCP prompt 目錄。 | | `fake_backend.py` | 184 | CI/無頭測試用的記憶體內假後端。 | @@ -724,7 +724,7 @@ socket server 有 8 MiB 讀取上限與 30 秒 handler timeout。 | `audit.py` | 78 | MCP 工具呼叫稽核記錄。 | | `context.py` | 71 | 傳給 opt-in 工具處理器的每次呼叫上下文。 | | `rate_limit.py` | 48 | 工具呼叫的 token bucket 限流。 | -| `__main__.py` | 87 | `je_auto_control_mcp` console script 進入點。 | +| `__main__.py` | 88 | `je_auto_control_mcp` console script 進入點。 | #### `utils/remote_desktop/`(11,990 行/56 檔) @@ -1025,9 +1025,9 @@ socket 預設綁 `127.0.0.1`;資源一律用 `with`。 | 層/子系統 | 檔案數 | 行數 | | --- | ---: | ---: | | `gui/` | 89 | 26,609 | -| `utils/mcp_server/` | 21 | 17,330 | +| `utils/mcp_server/` | 21 | 17,354 | | `utils/remote_desktop/` | 56 | 11,990 | -| `utils/executor/` | 6 | 9,075 | +| `utils/executor/` | 6 | 9,077 | | `utils/usb/` | 17 | 4,281 | | `je_auto_control/`(頂層 3 檔) | 3 | 2,363 | | `utils/accessibility/` | 13 | 2,835 | @@ -1045,5 +1045,5 @@ socket 預設綁 `127.0.0.1`;資源一律用 `with`。 | `autocontrol-lsp/` | 8 | 744 | | `utils/hotkey/` | 7 | 727 | | 其餘模組(約 286 個 `utils/` 子套件 + `android/`/`ios/`/周邊小工具) | 673 | 47,505 | -| **總計** | **1,025** | **140,727** | +| **總計** | **1,025** | **140,753** | diff --git a/je_auto_control/utils/executor/action_executor.py b/je_auto_control/utils/executor/action_executor.py index ee3cc775..5e5b8621 100644 --- a/je_auto_control/utils/executor/action_executor.py +++ b/je_auto_control/utils/executor/action_executor.py @@ -6711,8 +6711,10 @@ def _normalize_url(url: str, sort_query: bool = False, drop_fragment: bool = False) -> Dict[str, Any]: """Adapter: RFC 3986 syntax-based normalisation of a URL.""" from je_auto_control.utils.url_canon import normalize_url + # `drop_fragment` is the AC_ surface's name; url_canon calls the + # same flag `strip_fragment`. return {"url": normalize_url(url, sort_query=bool(sort_query), - drop_fragment=bool(drop_fragment))} + strip_fragment=bool(drop_fragment))} def _urls_equal(first: str, second: str) -> Dict[str, Any]: diff --git a/je_auto_control/utils/mcp_server/__main__.py b/je_auto_control/utils/mcp_server/__main__.py index bf6dc13f..bb26780f 100644 --- a/je_auto_control/utils/mcp_server/__main__.py +++ b/je_auto_control/utils/mcp_server/__main__.py @@ -7,6 +7,7 @@ import argparse import json import sys +from typing import Optional from je_auto_control.utils.mcp_server.fake_backend import ( install_fake_backend, maybe_install_from_env, @@ -50,7 +51,7 @@ def _build_parser() -> argparse.ArgumentParser: return parser -def main(argv: list = None) -> None: +def main(argv: Optional[list] = None) -> None: """CLI entry point. Performs the requested action and returns ``None``.""" parser = _build_parser() args = parser.parse_args(argv) @@ -72,13 +73,13 @@ def _print_listings(args: argparse.Namespace) -> None: sys.stdout, ensure_ascii=False, indent=2) sys.stdout.write("\n") if args.list_resources: - provider = default_resource_provider() - json.dump([resource.to_descriptor() for resource in provider.list()], + resources = default_resource_provider() + json.dump([resource.to_descriptor() for resource in resources.list()], sys.stdout, ensure_ascii=False, indent=2) sys.stdout.write("\n") if args.list_prompts: - provider = default_prompt_provider() - json.dump([prompt.to_descriptor() for prompt in provider.list()], + prompts = default_prompt_provider() + json.dump([prompt.to_descriptor() for prompt in prompts.list()], sys.stdout, ensure_ascii=False, indent=2) sys.stdout.write("\n") diff --git a/je_auto_control/utils/mcp_server/_client_requests.py b/je_auto_control/utils/mcp_server/_client_requests.py index 5d2e0119..1353845f 100644 --- a/je_auto_control/utils/mcp_server/_client_requests.py +++ b/je_auto_control/utils/mcp_server/_client_requests.py @@ -10,9 +10,12 @@ Destructive-tool confirmation lives here too: it is an elicitation round-trip, not a tool-execution step. """ +import itertools import json import threading -from typing import Any, Dict, List, Optional +from typing import ( + TYPE_CHECKING, Any, Callable, Dict, List, Optional, +) from je_auto_control.utils.logging.logging_instance import autocontrol_logger from je_auto_control.utils.mcp_server._protocol import ( @@ -29,6 +32,18 @@ class ClientRequestMixin: ``_outbound_id_counter`` and ``_sampling_id_counter``. """ + if TYPE_CHECKING: + # Declared, never defined: :class:`MCPServer` owns every one of + # these. The block is stripped at runtime, so nothing here can + # shadow what the host actually binds. + _writer: Optional[Callable[[str], None]] + _client_capabilities: Dict[str, Any] + _resources: Any + _outbound_lock: threading.Lock + _pending_outbound: Dict[Any, Dict[str, Any]] + _outbound_id_counter: "itertools.count[int]" + _sampling_id_counter: "itertools.count[int]" + @staticmethod def _is_outbound_response(method: Optional[str], msg_id: Any, message: Dict[str, Any]) -> bool: @@ -97,7 +112,7 @@ def _send_outbound_request(self, method: str, if writer is None: raise RuntimeError(f"{method} requires an outbound writer") request_id = f"srv-{next(self._outbound_id_counter)}" - slot = {"event": threading.Event()} + slot: Dict[str, Any] = {"event": threading.Event()} with self._outbound_lock: self._pending_outbound[request_id] = slot envelope = json.dumps({ @@ -159,7 +174,7 @@ def request_sampling(self, messages: List[Dict[str, Any]], params["systemPrompt"] = str(system_prompt) if model_preferences is not None: params["modelPreferences"] = dict(model_preferences) - slot = {"event": threading.Event()} + slot: Dict[str, Any] = {"event": threading.Event()} with self._outbound_lock: self._pending_outbound[request_id] = slot envelope = json.dumps({ diff --git a/je_auto_control/utils/mcp_server/_protocol.py b/je_auto_control/utils/mcp_server/_protocol.py index 7c7f1381..bb01272b 100644 --- a/je_auto_control/utils/mcp_server/_protocol.py +++ b/je_auto_control/utils/mcp_server/_protocol.py @@ -13,7 +13,7 @@ import subprocess # nosec B404 # reason: only its TimeoutExpired type is referenced import sys import time -from typing import Any, Dict, List, Optional +from typing import Any, Dict, List, Optional, Tuple, Type from je_auto_control.utils.exception.exceptions import AutoControlException from je_auto_control.utils.logging.logging_instance import autocontrol_logger @@ -39,8 +39,10 @@ _BUILTIN_DISPATCH_ERRORS = ( OSError, RuntimeError, ValueError, TypeError, KeyError, ) -_DISPATCH_ERRORS = _BUILTIN_DISPATCH_ERRORS + _FRAMEWORK_TOOL_ERRORS -_TOOL_INVOKE_ERRORS = ( +_DISPATCH_ERRORS: Tuple[Type[BaseException], ...] = ( + _BUILTIN_DISPATCH_ERRORS + _FRAMEWORK_TOOL_ERRORS +) +_TOOL_INVOKE_ERRORS: Tuple[Type[BaseException], ...] = ( _BUILTIN_DISPATCH_ERRORS + (AttributeError,) + _FRAMEWORK_TOOL_ERRORS ) diff --git a/je_auto_control/utils/mcp_server/server.py b/je_auto_control/utils/mcp_server/server.py index 5c24f943..7d51b1ff 100644 --- a/je_auto_control/utils/mcp_server/server.py +++ b/je_auto_control/utils/mcp_server/server.py @@ -7,6 +7,7 @@ message — no Content-Length framing — matching the MCP stdio spec. """ import contextlib +import functools import itertools import json import sys @@ -472,6 +473,8 @@ def _dispatch(self, msg_id: Any, method: Optional[str], params: Dict[str, Any]) -> Any: if method == _TOOLS_CALL_METHOD: return self._handle_tools_call(msg_id, params) + if method is None: + raise _MCPError(-32601, f"Method not found: {method}") nullary = { "ping": self._handle_ping, "tools/list": self._handle_tools_list, @@ -573,7 +576,8 @@ def _handle_resources_subscribe(self, if uri in self._resource_subscriptions: return {} handle = self._resources.subscribe( - uri, lambda u=uri: self._notify_resource_updated(u), + uri, + functools.partial(self._notify_resource_updated, uri), ) if handle is None: raise _MCPError(-32602, f"Unsubscribable resource: {uri}") diff --git a/je_auto_control/utils/mcp_server/tools/_handlers.py b/je_auto_control/utils/mcp_server/tools/_handlers.py index d5a5d39b..fa4a0b5b 100644 --- a/je_auto_control/utils/mcp_server/tools/_handlers.py +++ b/je_auto_control/utils/mcp_server/tools/_handlers.py @@ -1975,8 +1975,10 @@ def canonicalize_url(url): def normalize_url(url, sort_query=False, drop_fragment=False): from je_auto_control.utils.url_canon import normalize_url as _norm + # `drop_fragment` is this surface's name for it; url_canon calls the + # same flag `strip_fragment`. return {"url": _norm(url, sort_query=bool(sort_query), - drop_fragment=bool(drop_fragment))} + strip_fragment=bool(drop_fragment))} def urls_equal(first, second): @@ -2093,7 +2095,7 @@ def emit_event(event_type, data=None, source="je_auto_control", subject=None, url=None): from je_auto_control.utils.events import post_cloudevent, to_cloudevent event = to_cloudevent(event_type, source, data, subject=subject) - result = {"event": event} + result: Dict[str, Any] = {"event": event} if url: result["status"] = post_cloudevent(url, event) return result @@ -2160,7 +2162,7 @@ def jwt_decode(token, key, algorithms=None, audience=None, leeway=0.0): return {"ok": True, "claims": claims} -_RATE_LIMITERS = {} +_RATE_LIMITERS: Dict[str, Any] = {} _RATE_LIMITERS_LOCK = threading.Lock() @@ -3408,7 +3410,7 @@ def redact_pii(text, kinds=None, mode="label", mask_char="*"): def export_sarif(findings, path=None, tool_name="AutoControl"): from je_auto_control.utils.sarif import to_sarif, write_sarif - result = {"sarif": to_sarif(findings, tool_name=tool_name)} + result: Dict[str, Any] = {"sarif": to_sarif(findings, tool_name=tool_name)} if path: result["path"] = write_sarif(findings, path, tool_name=tool_name) return result diff --git a/je_auto_control/utils/webrunner_bridge/bridge.py b/je_auto_control/utils/webrunner_bridge/bridge.py index 6ed7b205..7f386e29 100644 --- a/je_auto_control/utils/webrunner_bridge/bridge.py +++ b/je_auto_control/utils/webrunner_bridge/bridge.py @@ -1,7 +1,7 @@ """Delegate ``WR_*`` browser-automation commands to ``je_web_runner``.""" from __future__ import annotations -from typing import Any, List, Mapping +from typing import Any, List, Mapping, Sequence class WebRunnerBridgeError(RuntimeError): @@ -70,7 +70,7 @@ def run_webrunner_action(action: Mapping[str, Any]) -> Any: ) from error -def run_webrunner_actions(actions: List[Mapping[str, Any]]) -> List[Any]: +def run_webrunner_actions(actions: Sequence[Mapping[str, Any]]) -> List[Any]: """Run a list of WR_* actions in order. Stops at the first error.""" if not isinstance(actions, list): raise WebRunnerBridgeError("actions must be a list") diff --git a/je_auto_control/wrapper/auto_control_keyboard.py b/je_auto_control/wrapper/auto_control_keyboard.py index 573ae657..ed5b9f3d 100644 --- a/je_auto_control/wrapper/auto_control_keyboard.py +++ b/je_auto_control/wrapper/auto_control_keyboard.py @@ -230,7 +230,7 @@ def write(write_string: str, is_shift: bool = False) -> Optional[str]: raise AutoControlKeyboardException(f"{keyboard_write_error_message} {repr(error)}") from error -def hotkey(key_code_list: list, is_shift: bool = False) -> Optional[Tuple[str, str]]: +def hotkey(key_code_list: list, is_shift: bool = False) -> Tuple[str, str]: """ 模擬組合鍵 (依序按下,再反向放開) Simulate hotkey (press all keys, then release in reverse order) diff --git a/je_auto_control/wrapper/auto_control_mouse.py b/je_auto_control/wrapper/auto_control_mouse.py index c936e1ff..376cf669 100644 --- a/je_auto_control/wrapper/auto_control_mouse.py +++ b/je_auto_control/wrapper/auto_control_mouse.py @@ -122,7 +122,7 @@ def get_mouse_position() -> tuple[int, int] | None: raise -def set_mouse_position(x: int, y: int) -> tuple[int, int] | None: +def set_mouse_position(x: int, y: int) -> tuple[int, int]: """ 設定滑鼠位置 Set mouse position diff --git a/test/verify/typing_contract_exempt.txt b/test/verify/typing_contract_exempt.txt index 55816b40..f6893401 100644 --- a/test/verify/typing_contract_exempt.txt +++ b/test/verify/typing_contract_exempt.txt @@ -14,7 +14,7 @@ # "Does not type-check" means on at least one of the three targets mypy can be # pointed at (win32, linux, darwin) — not just the one CI happens to run on. # -# Measured entries: 73 +# Measured entries: 69 je_auto_control.__main__ je_auto_control.cli @@ -53,10 +53,6 @@ je_auto_control.utils.http_client.http_client je_auto_control.utils.humanize.motion je_auto_control.utils.json_patch.json_patch je_auto_control.utils.locale_collation.locale_collation -je_auto_control.utils.mcp_server.__main__ -je_auto_control.utils.mcp_server._client_requests -je_auto_control.utils.mcp_server.server -je_auto_control.utils.mcp_server.tools._handlers je_auto_control.utils.monitor_layout.logical_frame je_auto_control.utils.mouse_path.mouse_path je_auto_control.utils.ocr.backends From b7735d97f547ef9236b8d491f3726424df2dc5ac Mon Sep 17 00:00:00 2001 From: JeffreyChen Date: Sat, 22 Aug 2026 21:16:24 +0800 Subject: [PATCH 15/19] Clear the long tail, and two invariants only the call graph knew MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Thirteen modules of two to six errors each. Three recurring shapes: `callable` used as a type (it is the builtin function, so mypy reads every call through it as calling something not callable — six times in plugin_loader, once in each hotkey backend); `x: SomeType = None` defaults; and tuples that lost their length, where `tuple(r)` and `cv2.boundingRect()` are `tuple[int, ...]` and the fields they feed promise four ints. Two are more than annotations: * `_StabilityTracker` read `now - self._since` on a path where `_since` is non-None only because of what an earlier call did. True today, invisible to a reader, one refactor from a TypeError inside the poll loop. It binds the value and treats "no start time" as "not stable". * `act_when_ready` passed `report.point` to a callback that requires a point. `point` is None whenever the target is invisible, and the guard above tests `report.actionable` — which implies visible, again only through the call graph. The point is checked where it is used. `text_regions` needed one justified suppression, and it is worth recording why: `cv2.MSER_create` exists at runtime in every supported OpenCV but is absent from the .pyi opencv-python ships (measured on 4.13.0). Progress.md now notes the consequence — the mypy config lists cv2 as a base dependency that ships no stubs, but it ships one, so the gate reads it and its verdict can move with the OpenCV version inside the >=4.8,<6 pin. --- Progress.md | 23 +++++++++--- WHATS_NEW.md | 37 +++++++++++++++++++ architecture_explore.md | 32 ++++++++-------- .../utils/actionability/actionability.py | 17 +++++---- .../utils/anchor_locator/locator.py | 27 ++++++++++++-- .../utils/flake_cluster/flake_cluster.py | 2 +- .../utils/hotkey/backends/linux_backend.py | 4 +- .../utils/hotkey/backends/windows_backend.py | 6 +-- .../utils/plugin_loader/plugin_loader.py | 14 +++---- je_auto_control/utils/redaction/policies.py | 4 +- je_auto_control/utils/redaction/rules.py | 20 +++++++--- .../utils/table_grid_fill/table_grid_fill.py | 4 +- .../utils/text_regions/text_regions.py | 12 ++++-- je_auto_control/utils/url_canon/url_canon.py | 6 ++- .../utils/window_capture/window_capture.py | 11 ++++-- .../utils/window_zorder/window_zorder.py | 8 ++-- test/verify/typing_contract_exempt.txt | 15 +------- 17 files changed, 160 insertions(+), 82 deletions(-) diff --git a/Progress.md b/Progress.md index 514137b3..f295fa4b 100644 --- a/Progress.md +++ b/Progress.md @@ -240,11 +240,11 @@ capability enum 值與 variadic `ei_seat_bind_capabilities`、event-type enum (`--cov-report=term-missing` 已經開著,CI 每一格都存了 `coverage.xml` artifact), 而不是齊頭式地補測試。 -### mypy:整包把關,69 個模組還沒過 +### mypy:整包把關,56 個模組還沒過 範圍不再是兩條路徑,而是**整包減去一張只准變少的清單** (`test/verify/typing_contract_exempt.txt`)。差別在於預設值:路徑清單只有人想到才會長, -新模組預設在圈外;現在新模組**預設就在契約裡**,1,017 個檔案有 948 個已經過關。 +新模組預設在圈外;現在新模組**預設就在契約裡**,1,017 個檔案有 961 個已經過關。 `wrapper` 那一群(6 個模組)在 2026-08-21 清掉了,做法見 [WHATS_NEW.md](WHATS_NEW.md): 平台縫的八個匯出名稱先宣告型別、再讓分支去綁定,其中三個用 @@ -326,10 +326,21 @@ Windows 後端在鎖定桌面上真的會回 `None`;`multi_language_wrapper` `TypeError`。對外的名字不動(動作 schema 與工具註冊表都用它),改的是兩個呼叫點。 做法見 [WHATS_NEW.md](WHATS_NEW.md)。 -剩下 69 個模組要清,而且**不再有大群集**:最大的檔案是 -`gui/remote_desktop/webrtc_panel.py`(27,見下),其餘每個檔都在 8 個錯誤以下 -(`utils/executor/action_executor.py` 8,`utils/table_grid_fill`/`utils/url_canon`/ -`utils/window_zorder`/`utils/plugin_loader` 各 6)。 +長尾那一批(13 個模組、每個 2~6 個錯誤)同日清掉,反覆出現的只有三種形狀: +`callable` 被當成型別用(`callable` 是內建函式)、`x: SomeType = None` 的隱含 +Optional、以及「掉了長度的 tuple」(`tuple(r)`/`cv2.boundingRect()` 都是 +`tuple[int, ...]`,而收它的欄位承諾四個 int)。另外抓到兩個只靠呼叫圖成立的不變量 +(`_StabilityTracker` 的 `now - self._since`、`act_when_ready` 把可能是 `None` 的 +`report.point` 傳給要求座標的 callback)。做法見 [WHATS_NEW.md](WHATS_NEW.md)。 + +**順帶一個 mypy 設定上的事實**:`pyproject.toml` 把 `cv2` 列在 +「ship no stubs 的基礎相依」底下,但 opencv-python **有**附 `.pyi`,所以閘門會去讀它, +而它的內容會隨 OpenCV 版本變動(實測 4.13.0:`cv2.MSER_create` 執行期存在、stub 裡沒有)。 +`>=4.8,<6` 這個範圍內版本一換,閘門的判定就可能跟著動——與 numpy 那條註解同一類的坑。 + +剩下 56 個模組要清,**不再有大群集**:最大的檔案是 +`gui/remote_desktop/webrtc_panel.py`(27,見下)與 +`utils/executor/action_executor.py`(8),其餘每個檔都只剩 1~2 個錯誤。 #### `webrtc_panel.py` 的型別豁免現在卡在它欠的那次拆檔 diff --git a/WHATS_NEW.md b/WHATS_NEW.md index cfc3abec..03dae6b3 100644 --- a/WHATS_NEW.md +++ b/WHATS_NEW.md @@ -208,6 +208,43 @@ comparison as a `DECIDE`, because the cleanest of them changes what the gate means rather than what the code says. **136 modules left, 881 of 1,017 files inside the contract.** +### Thirteen Small Modules, and a Stub That Disagrees With the Library + +Past the big clusters the list is a long tail: thirteen modules of two to six +errors each, cleared in one pass. Three recurring shapes, all of them cheap: + +- **`callable` used as a type.** It is the builtin *function*, so mypy reads + every call through the annotated value as calling something not callable. + `plugin_loader` had it six times, both hotkey backends once each. +- **`x: SomeType = None` defaults**, which PEP 484 prohibits and + `no_implicit_optional` rejects: the three `window_zorder` drivers. +- **A tuple that lost its length.** `tuple(r)` and `cv2.boundingRect(...)` are + `tuple[int, ...]`, and the fields they feed promise four ints. Spelling the + four out is both the fix and the documentation. + +Two are worth naming on their own: + +**`_StabilityTracker` read `now - self._since` on a path where `_since` could +only be non-`None` because of what an earlier call did.** True today, invisible +to a reader, and one refactor away from a `TypeError` in the poll loop. It +binds the value and treats "no start time" as "not stable yet". + +**`act_when_ready` passed `report.point` to a callback that requires a point.** +`point` is `None` whenever the target is invisible; the guard above it tests +`report.actionable`, which implies visible — again true, again only through the +call graph. The point is now checked where it is used. + +**And the cv2 stub disagrees with the cv2 that ships with it.** +`text_regions` calls `cv2.MSER_create`, which exists in every supported OpenCV +at runtime but is absent from the `.pyi` opencv-python installs (measured on +4.13.0: `hasattr(cv2, "MSER_create")` is `True`, the name is not in the stub). +That is one justified `type: ignore` — and a note for whoever next reads the +mypy config: cv2 is listed there under "base dependencies that ship no stubs", +but it does ship one, so the gate reads it and its verdict can move with the +OpenCV version inside the `>=4.8,<6` pin. + +**56 modules left, 961 of 1,017 files inside the contract.** + ### `normalize_url` Had Never Worked, on Either Surface The MCP cluster came off next, and the gate found a command that could not diff --git a/architecture_explore.md b/architecture_explore.md index d16a3560..a60297e1 100644 --- a/architecture_explore.md +++ b/architecture_explore.md @@ -20,7 +20,7 @@ iOS(WebDriverAgent)。核心能力是滑鼠/鍵盤控制、影像辨識、 | 指標 | 數值 | | --- | ---: | | Python 模組總數(含周邊子專案) | 1,031 | -| 程式碼總行數 | 140,818 | +| 程式碼總行數 | 140,861 | | `je_auto_control/utils/` 子套件數 | 310 | | `AC_*` 動作指令數(`known_commands()` 實測) | 773 | | 套件門面 `__all__` 公開名稱數 | 1,238 | @@ -338,7 +338,7 @@ socket server 有 8 MiB 讀取上限與 30 秒 handler timeout。 ### 5.4.4 輸入模擬與動作品質 -> 22 個套件、約 2,610 行。 +> 22 個套件、約 2,615 行。 | 模組 | 行數 | 職責 | | --- | ---: | --- | @@ -346,7 +346,7 @@ socket server 有 8 MiB 讀取上限與 30 秒 handler timeout。 | `utils/act_modes/` | 67 | actionability 閘門之上的 trial/force 動作模式 | | `utils/action_effect/` | 110 | 判定一個動作是否真的產生效果,並歸因到目標區域 | | `utils/action_grounding/` | 80 | 動作前的接地守衛(邊界檢查 + 吸附到元素) | -| `utils/actionability/` | 163 | 動作前就緒閘門(可見 + 穩定 + 啟用 + 未被遮擋) | +| `utils/actionability/` | 166 | 動作前就緒閘門(可見 + 穩定 + 啟用 + 未被遮擋) | | `utils/ensure_state/` | 72 | 冪等地把控制項/設定帶到期望狀態 | | `utils/field_entry/` | 76 | 清空再輸入的欄位填寫慣用法(Playwright `fill`) | | `utils/gamepad/` | 311 | 虛擬遊戲手把後端(Windows ViGEmBus 驅動) | @@ -358,7 +358,7 @@ socket server 有 8 MiB 讀取上限與 30 秒 handler timeout。 | `utils/mouse_relative/` | 59 | 相對位移滑鼠移動 | | `utils/postcondition/` | 138 | 宣告式的動作預期結果規格,對照畫面驗證 | | `utils/step_repair/` | 114 | 失敗/無效動作的修復策略(自我修正迴圈) | -| `utils/table_grid_fill/` | 141 | 以 OCR 文字填滿格線表格,取得可定址的表格 | +| `utils/table_grid_fill/` | 143 | 以 OCR 文字填滿格線表格,取得可定址的表格 | | `utils/input_reach/` | 111 | 送出去的輸入到不到得了:桌面鎖定查詢(免費)+ 實際送一個 F13 確認沒有被過濾(有副作用,只給診斷用) | | `utils/keyboard_layout/` | 148 | 向系統問「這個鍵盤配置下每個鍵印出什麼字」(`ToUnicodeEx`),問不到退回 US 對照表 | | `utils/text_unicode/` | 135 | 輸入任意 Unicode(emoji/CJK/重音字):優先送字元按鍵事件,不支援時退回剪貼簿貼上 | @@ -411,7 +411,7 @@ socket server 有 8 MiB 讀取上限與 30 秒 handler timeout。 ### 5.4.6 OCR 與文字理解 -> 19 個套件、約 3,180 行。 +> 19 個套件、約 3,184 行。 | 模組 | 行數 | 職責 | | --- | ---: | --- | @@ -432,7 +432,7 @@ socket server 有 8 MiB 讀取上限與 30 秒 handler timeout。 | `utils/text_blocks/` | 88 | 把 OCR 行組成段落與項目符號/編號清單 | | `utils/text_diff/` | 148 | unified diff 產生、套用與三方合併 | | `utils/text_normalize/` | 63 | Unicode 正規化與 slug 產生 | -| `utils/text_regions/` | 157 | 免模型的畫面文字區域偵測(MSER):區域與行 | +| `utils/text_regions/` | 161 | 免模型的畫面文字區域偵測(MSER):區域與行 | | `utils/text_similarity/` | 165 | 字串距離度量(文字比對用) | ### 5.4.7 無障礙樹與原生控制項 @@ -460,13 +460,13 @@ socket server 有 8 MiB 讀取上限與 30 秒 handler timeout。 ### 5.4.8 元素定位、自我修復與智慧等待 -> 23 個套件、約 3,995 行。 +> 23 個套件、約 4,014 行。 | 模組 | 行數 | 職責 | | --- | ---: | --- | | `utils/ab_locator/` | 336 | A/B 定位器框架:同時競速 N 種策略並記錄各自勝率 | | `utils/adaptive_timeout/` | 84 | 由觀測到的步驟耗時推導等待逾時,而非硬猜 | -| `utils/anchor_locator/` | 438 | 錨點定位器:以空間關係組合 影像/OCR/VLM/a11y 四種來源 | +| `utils/anchor_locator/` | 457 | 錨點定位器:以空間關係組合 影像/OCR/VLM/a11y 四種來源 | | `utils/app_idle/` | 108 | 等應用程式不再忙碌,再驅動下一步 | | `utils/change_localize/` | 80 | 把畫面變化歸因到實際改變的元素框 | | `utils/critic_features/` | 85 | 每步的 critic 特徵集合與規則式步驟評分 | @@ -523,7 +523,7 @@ socket server 有 8 MiB 讀取上限與 30 秒 handler timeout。 ### 5.4.11 伺服器、網路協定與外部整合 -> 24 個套件、約 5,917 行。 +> 24 個套件、約 5,919 行。 | 模組 | 行數 | 職責 | | --- | ---: | --- | @@ -549,7 +549,7 @@ socket server 有 8 MiB 讀取上限與 30 秒 handler timeout。 | `utils/socket_server/` | 131 | 執行 action JSON 的執行緒式 TCP 指令伺服器(預設綁 127.0.0.1) | | `utils/sse_client/` | 112 | Server-Sent Events 用戶端解析 | | `utils/tls_acme/` | 447 | TLS 自動化:HTTP-01 挑戰伺服器、金鑰/CSR、自動續期 | -| `utils/url_canon/` | 115 | RFC 3986 URL 正規化與查詢字串工具 | +| `utils/url_canon/` | 117 | RFC 3986 URL 正規化與查詢字串工具 | | `utils/webrunner_bridge/` | 161 | 把 action JSON 橋接到 WebRunner(`je_web_runner`) | ### 5.4.12 報表、可觀測性與測試治理 @@ -626,7 +626,7 @@ socket server 有 8 MiB 讀取上限與 30 秒 handler timeout。 ### 5.4.14 安全、機密與合規 -> 13 個套件、約 2,279 行。 +> 13 個套件、約 2,289 行。 | 模組 | 行數 | 職責 | | --- | ---: | --- | @@ -636,7 +636,7 @@ socket server 有 8 MiB 讀取上限與 30 秒 handler timeout。 | `utils/license_policy/` | 139 | 以 SBOM 元件評估 SPDX 授權允許/拒絕政策 | | `utils/provenance/` | 104 | SLSA 建置來源證明(in-toto v1) | | `utils/rbac/` | 272 | 角色型存取控制與逐使用者稽核歸因 | -| `utils/redaction/` | 457 | 截圖遮蔽層:規則偵測 + 政策 + 協調器(上傳 VLM 前先遮) | +| `utils/redaction/` | 467 | 截圖遮蔽層:規則偵測 + 政策 + 協調器(上傳 VLM 前先遮) | | `utils/sbom/` | 108 | SBOM(CycloneDX)產生 | | `utils/secret_ref/` | 126 | URI scheme 形式的值參照解析 | | `utils/secrets/` | 269 | 加密機密儲存庫,供 `${secrets.NAME}` 解析 | @@ -667,7 +667,7 @@ socket server 有 8 MiB 讀取上限與 30 秒 handler timeout。 ### 5.4.16 系統、視窗與剪貼簿 -> 16 個套件、約 2,412 行。 +> 16 個套件、約 2,415 行。 | 模組 | 行數 | 職責 | | --- | ---: | --- | @@ -683,7 +683,7 @@ socket server 有 8 MiB 讀取上限與 30 秒 handler timeout。 | `utils/shell_open/` | 97 | 以預設應用開啟檔案,或以預設瀏覽器開啟 URL | | `utils/system_volume/` | 194 | 讀取與控制系統主音量與靜音狀態 | | `utils/trash/` | 90 | 把檔案移到系統資源回收筒(可復原刪除) | -| `utils/window_capture/` | 255 | 逐視窗截圖、視窗版面儲存/還原、貼齊與排列 | +| `utils/window_capture/` | 258 | 逐視窗截圖、視窗版面儲存/還原、貼齊與排列 | | `utils/window_geometry/` | 81 | 視窗客戶區幾何(外框內縮、client→screen 對映) | | `utils/window_layout/` | 134 | 視窗拼貼/版面規劃器(左右半、四象限、網格、層疊) | | `utils/window_zorder/` | 76 | 視窗 z 序控制(最上層/移到最前/送到最後) | @@ -1044,6 +1044,6 @@ socket 預設綁 `127.0.0.1`;資源一律用 `with`。 | `osx/` | 17 | 915 | | `autocontrol-lsp/` | 8 | 744 | | `utils/hotkey/` | 7 | 727 | -| 其餘模組(約 286 個 `utils/` 子套件 + `android/`/`ios/`/周邊小工具) | 673 | 47,505 | -| **總計** | **1,025** | **140,753** | +| 其餘模組(約 286 個 `utils/` 子套件 + `android/`/`ios/`/周邊小工具) | 673 | 47,548 | +| **總計** | **1,025** | **140,796** | diff --git a/je_auto_control/utils/actionability/actionability.py b/je_auto_control/utils/actionability/actionability.py index 3389d8ac..bb23833d 100644 --- a/je_auto_control/utils/actionability/actionability.py +++ b/je_auto_control/utils/actionability/actionability.py @@ -85,11 +85,12 @@ def update(self, bbox: Optional[Bbox], now: float) -> bool: self._prev = object() return False token = (bbox, self._sampler(bbox) if self._sampler else None) - if token != self._prev: + since = self._since + if token != self._prev or since is None: self._prev = token self._since = now return False - return (now - self._since) >= self._stable_for_s + return (now - since) >= self._stable_for_s def _evaluate(bbox, tracker, now, enabled_probe, hit_tester): @@ -129,9 +130,10 @@ def wait_actionable(bbox_provider: BboxProvider, *, deadline = start + cfg.timeout_s while True: now = cfg.clock() - signals = _evaluate(bbox_provider(), tracker, now, enabled_probe, - hit_tester) - report = _report(*signals, now - start) + visible, stable, enabled, receives, point = _evaluate( + bbox_provider(), tracker, now, enabled_probe, hit_tester) + report = _report(visible, stable, enabled, receives, point, + now - start) if report.actionable or now >= deadline: return report cfg.sleep(cfg.poll_interval_s) @@ -150,7 +152,8 @@ def act_when_ready(action: Callable[[List[int]], Any], bbox_provider: BboxProvid report = wait_actionable(bbox_provider, region_sampler=region_sampler, enabled_probe=enabled_probe, hit_tester=hit_tester, config=config) - if not report.actionable: + point = report.point + if not report.actionable or point is None: raise AutoControlActionException( f"target not actionable ({report.reason}) after {report.waited_s}s") - return action(report.point) + return action(point) diff --git a/je_auto_control/utils/anchor_locator/locator.py b/je_auto_control/utils/anchor_locator/locator.py index 79afdf32..260d0add 100644 --- a/je_auto_control/utils/anchor_locator/locator.py +++ b/je_auto_control/utils/anchor_locator/locator.py @@ -85,12 +85,30 @@ def image_locator(template_path: str, detect_threshold=float(detect_threshold)) +def _as_region(region: Optional[List[int]] + ) -> Optional[Tuple[int, int, int, int]]: + """Return a four-tuple region, or None when there is no region.""" + if not region: + return None + if len(region) != 4: + raise ValueError(f"region must have 4 values, got {len(region)}") + left, top, width, height = (int(value) for value in region) + return left, top, width, height + + +def _require(value: Optional[str], locator: "Locator", field: str) -> str: + """Return a field this locator kind must carry, or say which is missing.""" + if value is None: + raise ValueError(f"{locator.kind} locator has no {field}") + return value + + def ocr_locator(text: str, *, min_confidence: float = 60.0, region: Optional[List[int]] = None) -> Locator: return Locator( kind=KIND_OCR, text=str(text), min_confidence=float(min_confidence), - region=tuple(region) if region else None, + region=_as_region(region), ) @@ -342,7 +360,7 @@ def _ocr_center(locator: Locator) -> Optional[Tuple[int, int]]: try: from je_auto_control.utils.ocr.ocr_engine import locate_text_center return locate_text_center( - locator.text, + _require(locator.text, locator, "text"), region=list(locator.region) if locator.region else None, min_confidence=locator.min_confidence, ) @@ -354,7 +372,7 @@ def _ocr_candidates(locator: Locator) -> List[_Bbox]: try: from je_auto_control.utils.ocr.ocr_engine import find_text_matches matches = find_text_matches( - locator.text, + _require(locator.text, locator, "text"), region=list(locator.region) if locator.region else None, min_confidence=locator.min_confidence, ) @@ -368,7 +386,8 @@ def _vlm_point(locator: Locator) -> Optional[Tuple[int, int]]: try: from je_auto_control.utils.vision.vlm_api import locate_by_description return locate_by_description( - locator.description, model=locator.model, + _require(locator.description, locator, "description"), + model=locator.model, ) except _LOCATE_ERRORS: return None diff --git a/je_auto_control/utils/flake_cluster/flake_cluster.py b/je_auto_control/utils/flake_cluster/flake_cluster.py index d35d261c..f727c111 100644 --- a/je_auto_control/utils/flake_cluster/flake_cluster.py +++ b/je_auto_control/utils/flake_cluster/flake_cluster.py @@ -36,7 +36,7 @@ def cofailure_pairs(runs: Sequence[Sequence[str]], *, Each entry is ``{tests:[a,b], jaccard, co_failures}``, most similar first. """ fails = _fail_runs(runs) - pairs = [] + pairs: List[Dict[str, Any]] = [] for left, right in combinations(sorted(fails), 2): score = _set_jaccard(fails[left], fails[right]) if score >= float(threshold): diff --git a/je_auto_control/utils/hotkey/backends/linux_backend.py b/je_auto_control/utils/hotkey/backends/linux_backend.py index 850a705c..2066d17a 100644 --- a/je_auto_control/utils/hotkey/backends/linux_backend.py +++ b/je_auto_control/utils/hotkey/backends/linux_backend.py @@ -4,7 +4,7 @@ key, matching Windows ``RegisterHotKey`` semantics. NumLock / CapsLock are masked so the hotkey still fires with those toggles active. """ -from typing import Dict, List, Optional, Tuple +from typing import Callable, Dict, List, Optional, Tuple from je_auto_control.utils.hotkey.backends.base import HotkeyBackend from je_auto_control.utils.hotkey.hotkey_daemon import ( @@ -155,7 +155,7 @@ def _grab_masked(root, binding: HotkeyBinding, grabbed.append(extra_mask) return True - def _drain(self, disp, fire: "callable") -> None: + def _drain(self, disp, fire: Callable[[str], None]) -> None: from Xlib import X while disp.pending_events(): diff --git a/je_auto_control/utils/hotkey/backends/windows_backend.py b/je_auto_control/utils/hotkey/backends/windows_backend.py index 68a6f669..90b8f251 100644 --- a/je_auto_control/utils/hotkey/backends/windows_backend.py +++ b/je_auto_control/utils/hotkey/backends/windows_backend.py @@ -1,5 +1,5 @@ """Windows hotkey backend: ``RegisterHotKey`` + a message-pump thread.""" -from typing import Dict, List, Optional, Tuple +from typing import Callable, Dict, List, Optional, Tuple from je_auto_control.utils.hotkey.backends.base import HotkeyBackend from je_auto_control.utils.hotkey.hotkey_daemon import ( @@ -22,7 +22,7 @@ def run_forever(self, context: BackendContext) -> None: import ctypes from ctypes import wintypes - user32 = ctypes.WinDLL("user32", use_last_error=True) + user32 = ctypes.WinDLL("user32", use_last_error=True) # type: ignore[attr-defined] # reason: win32-only ctypes user32.RegisterHotKey.argtypes = [ wintypes.HWND, ctypes.c_int, wintypes.UINT, wintypes.UINT, ] @@ -85,7 +85,7 @@ def _try_register(self, user32, binding: HotkeyBinding) -> None: ) def _dispatch(self, registered_id: int, - fire: "callable") -> None: + fire: Callable[[str], None]) -> None: match: Optional[str] = None for bid, (reg_id, _combo) in self._registered.items(): if reg_id == registered_id: diff --git a/je_auto_control/utils/plugin_loader/plugin_loader.py b/je_auto_control/utils/plugin_loader/plugin_loader.py index 456282db..febbcea5 100644 --- a/je_auto_control/utils/plugin_loader/plugin_loader.py +++ b/je_auto_control/utils/plugin_loader/plugin_loader.py @@ -13,12 +13,12 @@ import pathlib import uuid from types import ModuleType -from typing import Dict, List +from typing import Any, Callable, Dict, List from je_auto_control.utils.logging.logging_instance import autocontrol_logger -def load_plugin_file(path: str) -> Dict[str, callable]: +def load_plugin_file(path: str) -> Dict[str, Callable[..., Any]]: """Import ``path`` and return a mapping of ``AC_*`` callables it defines.""" resolved = os.path.realpath(path) if not os.path.isfile(resolved): @@ -27,12 +27,12 @@ def load_plugin_file(path: str) -> Dict[str, callable]: return discover_plugin_commands(module) -def load_plugin_directory(directory: str) -> Dict[str, callable]: +def load_plugin_directory(directory: str) -> Dict[str, Callable[..., Any]]: """Load every ``*.py`` in ``directory`` and merge their AC_* callables.""" root = pathlib.Path(os.path.realpath(directory)) if not root.is_dir(): raise NotADirectoryError(f"plugin directory not found: {root}") - merged: Dict[str, callable] = {} + merged: Dict[str, Callable[..., Any]] = {} for file_path in sorted(root.glob("*.py")): if file_path.name.startswith("_"): continue @@ -46,9 +46,9 @@ def load_plugin_directory(directory: str) -> Dict[str, callable]: return merged -def discover_plugin_commands(module: ModuleType) -> Dict[str, callable]: +def discover_plugin_commands(module: ModuleType) -> Dict[str, Callable[..., Any]]: """Return every ``AC_*`` callable defined on ``module``.""" - commands: Dict[str, callable] = {} + commands: Dict[str, Callable[..., Any]] = {} for attr_name in dir(module): if not attr_name.startswith("AC_"): continue @@ -58,7 +58,7 @@ def discover_plugin_commands(module: ModuleType) -> Dict[str, callable]: return commands -def register_plugin_commands(commands: Dict[str, callable]) -> List[str]: +def register_plugin_commands(commands: Dict[str, Callable[..., Any]]) -> List[str]: """Register ``commands`` into the global executor and return their names.""" from je_auto_control.utils.executor.action_executor import executor for name, func in commands.items(): diff --git a/je_auto_control/utils/redaction/policies.py b/je_auto_control/utils/redaction/policies.py index 329bec6f..f5ab15ca 100644 --- a/je_auto_control/utils/redaction/policies.py +++ b/je_auto_control/utils/redaction/policies.py @@ -54,7 +54,9 @@ def with_extra_regions( """Return a policy with ``extras`` appended to ``regions``.""" return RedactionPolicy( detectors=tuple(self.detectors), - regions=tuple(self.regions) + tuple(tuple(r) for r in extras), + regions=tuple(self.regions) + tuple( + (int(r[0]), int(r[1]), int(r[2]), int(r[3])) + for r in extras), blur_radius=self.blur_radius, overlay_color=self.overlay_color, ) diff --git a/je_auto_control/utils/redaction/rules.py b/je_auto_control/utils/redaction/rules.py index 9f171aa4..0a0f70e7 100644 --- a/je_auto_control/utils/redaction/rules.py +++ b/je_auto_control/utils/redaction/rules.py @@ -13,7 +13,7 @@ from __future__ import annotations import re -from typing import Any, Callable, Dict, Iterable, List, Tuple +from typing import Any, Callable, Dict, Iterable, List, Mapping, Tuple from je_auto_control.utils.redaction.policies import ( DETECTOR_CREDIT_CARD, DETECTOR_EMAIL, DETECTOR_SECURE_FIELD, @@ -144,14 +144,22 @@ def _overlap(a: BoundingBox, b: BoundingBox) -> bool: or a[3] < b[1] or b[3] < a[1]) -def _normalise_bbox(bbox) -> BoundingBox: +def _first_int(source: Mapping[str, Any], *names: str, default: int = 0) -> int: + """Return the first of ``names`` present in ``source``, as an int.""" + for name in names: + if name in source: + return int(source[name]) + return default + + +def _normalise_bbox(bbox: Any) -> BoundingBox: if bbox is None: raise ValueError("bbox cannot be None") if isinstance(bbox, dict): - x1 = int(bbox.get("x1", bbox.get("left", 0))) - y1 = int(bbox.get("y1", bbox.get("top", 0))) - x2 = int(bbox.get("x2", bbox.get("right", x1))) - y2 = int(bbox.get("y2", bbox.get("bottom", y1))) + x1 = _first_int(bbox, "x1", "left") + y1 = _first_int(bbox, "y1", "top") + x2 = _first_int(bbox, "x2", "right", default=x1) + y2 = _first_int(bbox, "y2", "bottom", default=y1) else: seq = list(bbox) if len(seq) != 4: diff --git a/je_auto_control/utils/table_grid_fill/table_grid_fill.py b/je_auto_control/utils/table_grid_fill/table_grid_fill.py index d5ca4534..0cbaf808 100644 --- a/je_auto_control/utils/table_grid_fill/table_grid_fill.py +++ b/je_auto_control/utils/table_grid_fill/table_grid_fill.py @@ -100,7 +100,9 @@ def _spans(grid: Dict[str, Any], text_boxes: Sequence[Box]) -> List[Dict[str, An left, top, right, bottom = _box_bounds(box) c0, c1 = _index_of(left, col_spans), _index_of(right - 1, col_spans) r0, r1 = _index_of(top, row_spans), _index_of(bottom - 1, row_spans) - if None in (c0, c1, r0, r1) or (c0 == c1 and r0 == r1): + if c0 is None or c1 is None or r0 is None or r1 is None: + continue + if c0 == c1 and r0 == r1: continue found.append({"row": r0, "col": c0, "row_span": r1 - r0 + 1, "col_span": c1 - c0 + 1, "text": str(box.get("text", ""))}) diff --git a/je_auto_control/utils/text_regions/text_regions.py b/je_auto_control/utils/text_regions/text_regions.py index 8ba949cb..ed5a0206 100644 --- a/je_auto_control/utils/text_regions/text_regions.py +++ b/je_auto_control/utils/text_regions/text_regions.py @@ -44,8 +44,8 @@ def _accept(rect: Rect, shape, min_area: int, max_area: Optional[int], # that OpenCV 4 segmented fine now yield zero regions at the default # min_diversity (0.2). Relax the pruning progressively before concluding # the frame has no text. -_MSER_PARAM_LADDER = ({}, {"min_diversity": 0.01}, - {"delta": 1, "min_diversity": 0.0}) +_MSER_PARAM_LADDER: Tuple[Dict[str, Any], ...] = ( + {}, {"min_diversity": 0.01}, {"delta": 1, "min_diversity": 0.0}) def _detect_regions(gray): @@ -53,7 +53,10 @@ def _detect_regions(gray): import cv2 regions = () for params in _MSER_PARAM_LADDER: - regions, _bboxes = cv2.MSER_create(**params).detectRegions(gray) + # `MSER_create` is present at runtime in every supported OpenCV + # but is not declared in the stub opencv-python ships. + mser = cv2.MSER_create(**params) # type: ignore[attr-defined] # reason: absent from the cv2 stub + regions, _bboxes = mser.detectRegions(gray) if len(regions): break return regions @@ -67,7 +70,8 @@ def _filtered_boxes(gray, min_area: int, max_area: Optional[int], out: List[Rect] = [] seen = set() for points in regions: - rect = cv2.boundingRect(points.reshape(-1, 1, 2)) + box = cv2.boundingRect(points.reshape(-1, 1, 2)) + rect: Rect = (int(box[0]), int(box[1]), int(box[2]), int(box[3])) if rect not in seen and _accept(rect, gray.shape, min_area, max_area, max_aspect): out.append(rect) diff --git a/je_auto_control/utils/url_canon/url_canon.py b/je_auto_control/utils/url_canon/url_canon.py index 9889280d..fbcd9c7a 100644 --- a/je_auto_control/utils/url_canon/url_canon.py +++ b/je_auto_control/utils/url_canon/url_canon.py @@ -13,7 +13,9 @@ import posixpath import re from typing import List, Mapping, Sequence, Tuple, Union -from urllib.parse import parse_qsl, urlencode, urlsplit, urlunsplit +from urllib.parse import ( + SplitResult, parse_qsl, urlencode, urlsplit, urlunsplit, +) _DEFAULT_PORTS = {"http": 80, "https": 443, "ftp": 21, "ws": 80, "wss": 443} _PERCENT = re.compile(r"%[0-9a-fA-F]{2}") @@ -37,7 +39,7 @@ def _normalize_path(path: str) -> str: return _normalize_percent(collapsed) -def _build_netloc(parts: "urlsplit", host: str, scheme: str, +def _build_netloc(parts: SplitResult, host: str, scheme: str, strip_default_port: bool) -> str: """Reassemble the authority, dropping a redundant default port.""" userinfo = "" diff --git a/je_auto_control/utils/window_capture/window_capture.py b/je_auto_control/utils/window_capture/window_capture.py index 30fdf3d2..d75de259 100644 --- a/je_auto_control/utils/window_capture/window_capture.py +++ b/je_auto_control/utils/window_capture/window_capture.py @@ -41,7 +41,8 @@ def _win32_geometry(hwnd: int) -> Optional[Rect]: import ctypes from ctypes import wintypes rect = wintypes.RECT() - if not ctypes.windll.user32.GetWindowRect(hwnd, ctypes.byref(rect)): + user32 = ctypes.windll.user32 # type: ignore[attr-defined] # reason: win32-only ctypes + if not user32.GetWindowRect(hwnd, ctypes.byref(rect)): return None return (rect.left, rect.top, rect.right - rect.left, rect.bottom - rect.top) @@ -124,11 +125,13 @@ def restore_window_layout(layout: Union[List[Dict[str, Any]], str, Path], *, ``layout`` is a list from :func:`save_window_layout`, or a path to the JSON it wrote. ``mover`` is injectable for tests. """ - if isinstance(layout, (str, Path)): - layout = json.loads(Path(layout).read_text(encoding="utf-8")) + entries: List[Dict[str, Any]] = ( + json.loads(Path(layout).read_text(encoding="utf-8")) + if isinstance(layout, (str, Path)) else list(layout) + ) move = mover or _default_mover restored = 0 - for entry in layout: + for entry in entries: title = entry.get("title") if title and move(title, int(entry["x"]), int(entry["y"]), int(entry["width"]), int(entry["height"])): diff --git a/je_auto_control/utils/window_zorder/window_zorder.py b/je_auto_control/utils/window_zorder/window_zorder.py index b228ee29..f7b0726f 100644 --- a/je_auto_control/utils/window_zorder/window_zorder.py +++ b/je_auto_control/utils/window_zorder/window_zorder.py @@ -12,7 +12,7 @@ (returning ``False`` on other platforms). Imports no ``PySide6``. """ import sys -from typing import Any, Callable, Dict +from typing import Any, Callable, Dict, Optional ZOrderDriver = Callable[[str, str], bool] @@ -52,18 +52,18 @@ def _default_driver(title: str, action: str) -> bool: def set_topmost(title: str, on: bool = True, *, - driver: Callable[[str, str], bool] = None) -> bool: + driver: Optional[Callable[[str, str], bool]] = None) -> bool: """Pin the window matching ``title`` always-on-top (or release it when ``on`` is False).""" return (driver or _default_driver)(title, "topmost" if on else "notopmost") def bring_to_front(title: str, *, - driver: Callable[[str, str], bool] = None) -> bool: + driver: Optional[Callable[[str, str], bool]] = None) -> bool: """Raise the window matching ``title`` to the top of the z-order.""" return (driver or _default_driver)(title, "top") def send_to_back(title: str, *, - driver: Callable[[str, str], bool] = None) -> bool: + driver: Optional[Callable[[str, str], bool]] = None) -> bool: """Send the window matching ``title`` to the bottom of the z-order.""" return (driver or _default_driver)(title, "bottom") diff --git a/test/verify/typing_contract_exempt.txt b/test/verify/typing_contract_exempt.txt index f6893401..059fd348 100644 --- a/test/verify/typing_contract_exempt.txt +++ b/test/verify/typing_contract_exempt.txt @@ -14,16 +14,14 @@ # "Does not type-check" means on at least one of the three targets mypy can be # pointed at (win32, linux, darwin) — not just the one CI happens to run on. # -# Measured entries: 69 +# Measured entries: 56 je_auto_control.__main__ je_auto_control.cli je_auto_control.gui.remote_desktop.webrtc_panel je_auto_control.utils.act_modes.act_modes -je_auto_control.utils.actionability.actionability je_auto_control.utils.admin.admin_client je_auto_control.utils.agent_memory.agent_memory -je_auto_control.utils.anchor_locator.locator je_auto_control.utils.annotate.annotate je_auto_control.utils.bulkhead.bulkhead je_auto_control.utils.change_localize.change_localize @@ -43,12 +41,9 @@ je_auto_control.utils.element_proposal.element_proposal je_auto_control.utils.executor.action_executor je_auto_control.utils.expect_poll.expect_poll je_auto_control.utils.feature_match.feature_match -je_auto_control.utils.flake_cluster.flake_cluster je_auto_control.utils.flow_debugger.flow_debugger je_auto_control.utils.generate_report.generate_xml_report je_auto_control.utils.governance.credential_broker -je_auto_control.utils.hotkey.backends.linux_backend -je_auto_control.utils.hotkey.backends.windows_backend je_auto_control.utils.http_client.http_client je_auto_control.utils.humanize.motion je_auto_control.utils.json_patch.json_patch @@ -56,14 +51,11 @@ je_auto_control.utils.locale_collation.locale_collation je_auto_control.utils.monitor_layout.logical_frame je_auto_control.utils.mouse_path.mouse_path je_auto_control.utils.ocr.backends -je_auto_control.utils.plugin_loader.plugin_loader je_auto_control.utils.preprocess.preprocess je_auto_control.utils.profiler.resource_profiler je_auto_control.utils.project.create_project_structure je_auto_control.utils.qr.qr je_auto_control.utils.rate_limit.rate_limit -je_auto_control.utils.redaction.policies -je_auto_control.utils.redaction.rules je_auto_control.utils.rotated_match.rotated_match je_auto_control.utils.run_history.history_store je_auto_control.utils.sbom.sbom @@ -73,15 +65,10 @@ je_auto_control.utils.search_index.search_index je_auto_control.utils.secrets.secret_store je_auto_control.utils.socket_server.auto_control_socket_server je_auto_control.utils.soft_assert.soft_assert -je_auto_control.utils.table_grid_fill.table_grid_fill je_auto_control.utils.test_record.record_test_class je_auto_control.utils.text_normalize.text_normalize -je_auto_control.utils.text_regions.text_regions je_auto_control.utils.tls_acme.challenge -je_auto_control.utils.url_canon.url_canon je_auto_control.utils.visual_regression.compare je_auto_control.utils.watcher.watcher -je_auto_control.utils.window_capture.window_capture -je_auto_control.utils.window_zorder.window_zorder je_auto_control.utils.work_queue.work_queue je_auto_control.utils.xml.xml_file.xml_file From 4fa556e1cfc581f5cd9ccdbb76d2b9c2c206c79e Mon Sep 17 00:00:00 2001 From: JeffreyChen Date: Sat, 22 Aug 2026 21:21:53 +0800 Subject: [PATCH 16/19] Clear the action executor off the typing contract MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Eight errors, and two of them were the dispatch table disagreeing with itself. `_android_client_cache` is keyed by the `(serial, adb_path)` pair but declared `Dict[Optional[str], Any]`, and `backend_obj` was assigned an Anthropic backend on one branch and an OpenAI one on the other, so mypy read the second as an error rather than as the sibling implementation of `AgentBackend` it is. `_unwrap_action_list` reassigned its own parameter to the result of `dict.get`, widening a `Union[list, dict]` to include None; it unwraps through a local now, which is also what the `isinstance` checks below it were already assuming. `ensure_state`'s `StateSetter` promised `Callable[[Any], None]` while every real setter in the tree returns a bool the function then ignores — it re-reads instead of trusting what a setter reports, which is the whole point of the helper. The alias says `Any` now, and the comment says why. `RedactionPolicy.with_extra_regions` likewise: it coerces its extras to four ints internally, so it can accept the sequences callers actually have. --- architecture_explore.md | 20 +++++++++---------- .../utils/ensure_state/ensure_state.py | 4 +++- .../utils/executor/action_executor.py | 20 +++++++++++-------- je_auto_control/utils/redaction/policies.py | 4 ++-- test/verify/typing_contract_exempt.txt | 3 +-- 5 files changed, 28 insertions(+), 23 deletions(-) diff --git a/architecture_explore.md b/architecture_explore.md index a60297e1..29200b4d 100644 --- a/architecture_explore.md +++ b/architecture_explore.md @@ -20,7 +20,7 @@ iOS(WebDriverAgent)。核心能力是滑鼠/鍵盤控制、影像辨識、 | 指標 | 數值 | | --- | ---: | | Python 模組總數(含周邊子專案) | 1,031 | -| 程式碼總行數 | 140,861 | +| 程式碼總行數 | 140,867 | | `je_auto_control/utils/` 子套件數 | 310 | | `AC_*` 動作指令數(`known_commands()` 實測) | 773 | | 套件門面 `__all__` 公開名稱數 | 1,238 | @@ -268,7 +268,7 @@ socket server 有 8 MiB 讀取上限與 30 秒 handler timeout。 ### 5.4.1 執行引擎與腳本資產 -> 24 個套件、約 12,902 行。 +> 24 個套件、約 12,906 行。 | 模組 | 行數 | 職責 | | --- | ---: | --- | @@ -279,7 +279,7 @@ socket server 有 8 MiB 讀取上限與 30 秒 handler timeout。 | `utils/dag/` | 475 | 跨主機 DAG 編排器(圖模型 + runner) | | `utils/decision_table/` | 103 | DMN 風格決策表:規則 + 命中策略,把分支外部化 | | `utils/deterministic/` | 96 | 決定性執行控制:固定亂數種子 + 凍結時鐘 | -| `utils/executor/` | 9,077 | **核心**。`Executor` 指令分派表(773 個 `AC_*`)、參數插值、乾跑、逐步 callback;`flow_control` 提供 34 個區塊指令(迴圈/分支/try/巨集/變數) | +| `utils/executor/` | 9,081 | **核心**。`Executor` 指令分派表(773 個 `AC_*`)、參數插值、乾跑、逐步 callback;`flow_control` 提供 34 個區塊指令(迴圈/分支/try/巨集/變數) | | `utils/flow_debugger/` | 136 | action list 的單步除錯器與追蹤器 | | `utils/input_macro/` | 342 | 定時輸入事件:錄製結果的整形(`timeline`/`InputRecorder`,Windows 與 macOS 共用)、重播與宣告式輸入序列 DSL | | `utils/json/` | 74 | action JSON 檔讀寫與正規化格式化(`fmt --check` 的後端) | @@ -338,7 +338,7 @@ socket server 有 8 MiB 讀取上限與 30 秒 handler timeout。 ### 5.4.4 輸入模擬與動作品質 -> 22 個套件、約 2,615 行。 +> 22 個套件、約 2,617 行。 | 模組 | 行數 | 職責 | | --- | ---: | --- | @@ -347,7 +347,7 @@ socket server 有 8 MiB 讀取上限與 30 秒 handler timeout。 | `utils/action_effect/` | 110 | 判定一個動作是否真的產生效果,並歸因到目標區域 | | `utils/action_grounding/` | 80 | 動作前的接地守衛(邊界檢查 + 吸附到元素) | | `utils/actionability/` | 166 | 動作前就緒閘門(可見 + 穩定 + 啟用 + 未被遮擋) | -| `utils/ensure_state/` | 72 | 冪等地把控制項/設定帶到期望狀態 | +| `utils/ensure_state/` | 74 | 冪等地把控制項/設定帶到期望狀態 | | `utils/field_entry/` | 76 | 清空再輸入的欄位填寫慣用法(Playwright `fill`) | | `utils/gamepad/` | 311 | 虛擬遊戲手把後端(Windows ViGEmBus 驅動) | | `utils/humanize/` | 183 | 擬人輸入:貝茲曲線滑鼠路徑 + 抖動打字節奏 | @@ -692,11 +692,11 @@ socket server 有 8 MiB 讀取上限與 30 秒 handler timeout。 上表以子套件為單位;以下把行數最大的幾個子系統展開到檔案層。 -#### `utils/executor/`(9,077 行)— 執行核心 +#### `utils/executor/`(9,081 行)— 執行核心 | 檔案 | 行數 | 職責 | | --- | ---: | --- | -| `action_executor.py` | 8,127 | `Executor` 類別與 `event_dict` 分派表(773 個指令),另含數百個把 utils 能力接成指令的 adapter 函式;全域單例 `executor` 與 `add_command_to_executor()` 擴充點。 | +| `action_executor.py` | 8,131 | `Executor` 類別與 `event_dict` 分派表(773 個指令),另含數百個把 utils 能力接成指令的 adapter 函式;全域單例 `executor` 與 `add_command_to_executor()` 擴充點。 | | `flow_control.py` | 530 | 真正的流程控制:`AC_loop`/`AC_for_each`/`AC_while_*`/`AC_if_*`/`AC_try`/`AC_retry`/`AC_parallel`/`AC_define_macro`/`AC_call_macro`/變數指令(`AC_set_var`/`AC_get_var`/`AC_inc_var`)。`LoopBreak`/`LoopContinue` 以例外實作。34 個區塊指令的分派表 `BLOCK_COMMANDS` 也在這裡,含下一列匯入的資料來源指令。 | | `flow_data_commands.py` | 253 | `AC_*_to_var` 資料來源與轉換指令:shell、時鐘、亂數、PDF、TOTP、SQL、檔案、HTTP、OCR,加上 `AC_assert_var`/`AC_assert_db`/`AC_assert_duration`/`AC_transform_var`。都不執行巢狀 action list,所以沒有迴圈/分支語意。 | | `action_schema.py` | 128 | action list 的結構驗證:形狀、參數型別、未知指令拒絕。單一走訪同時支援兩種消費方式:`validate_actions()` 遇到第一個問題就拋、`unknown_command_names()` 收齊全部不認得的名字(REST `/execute` 用它回 400)。 | @@ -1027,7 +1027,7 @@ socket 預設綁 `127.0.0.1`;資源一律用 `with`。 | `gui/` | 89 | 26,609 | | `utils/mcp_server/` | 21 | 17,354 | | `utils/remote_desktop/` | 56 | 11,990 | -| `utils/executor/` | 6 | 9,077 | +| `utils/executor/` | 6 | 9,081 | | `utils/usb/` | 17 | 4,281 | | `je_auto_control/`(頂層 3 檔) | 3 | 2,363 | | `utils/accessibility/` | 13 | 2,835 | @@ -1044,6 +1044,6 @@ socket 預設綁 `127.0.0.1`;資源一律用 `with`。 | `osx/` | 17 | 915 | | `autocontrol-lsp/` | 8 | 744 | | `utils/hotkey/` | 7 | 727 | -| 其餘模組(約 286 個 `utils/` 子套件 + `android/`/`ios/`/周邊小工具) | 673 | 47,548 | -| **總計** | **1,025** | **140,796** | +| 其餘模組(約 286 個 `utils/` 子套件 + `android/`/`ios/`/周邊小工具) | 673 | 47,550 | +| **總計** | **1,025** | **140,802** | diff --git a/je_auto_control/utils/ensure_state/ensure_state.py b/je_auto_control/utils/ensure_state/ensure_state.py index 0a703590..64ba91e3 100644 --- a/je_auto_control/utils/ensure_state/ensure_state.py +++ b/je_auto_control/utils/ensure_state/ensure_state.py @@ -21,7 +21,9 @@ from typing import Any, Callable, Dict StateReader = Callable[[], Any] -StateSetter = Callable[[Any], None] +#: The return value is ignored — `ensure_state` re-reads instead of +#: trusting what a setter reports. +StateSetter = Callable[[Any], Any] StateEquals = Callable[[Any, Any], bool] diff --git a/je_auto_control/utils/executor/action_executor.py b/je_auto_control/utils/executor/action_executor.py index 5e5b8621..4aae98b3 100644 --- a/je_auto_control/utils/executor/action_executor.py +++ b/je_auto_control/utils/executor/action_executor.py @@ -1,5 +1,5 @@ import types -from typing import Any, Callable, Dict, List, Optional, Sequence, Union +from typing import Any, Callable, Dict, List, Optional, Sequence, Tuple, Union from je_auto_control.utils.exception.exception_tags import ( action_is_null_error_message, add_command_exception_error_message, @@ -702,6 +702,7 @@ def _run_agent(goal: str, via :func:`_computer_use` / ``AC_computer_use``. """ from je_auto_control.utils.agent import AgentBudget, AgentLoop + from je_auto_control.utils.agent.agent_loop import AgentBackend from je_auto_control.utils.agent.backends import ( AnthropicAgentBackend, OpenAIAgentBackend, ) @@ -709,6 +710,7 @@ def _run_agent(goal: str, export_anthropic_tools, export_openai_tools, ) name = (backend or "anthropic").strip().lower() + backend_obj: AgentBackend if name == "anthropic": tools = export_anthropic_tools() backend_obj = AnthropicAgentBackend( @@ -1612,7 +1614,7 @@ def _ac_web_current_url() -> Any: # --- Android via ADB (Phase 9.7) --------------------------------------- -_android_client_cache: Dict[Optional[str], Any] = {} +_android_client_cache: Dict[Tuple[Optional[str], Optional[str]], Any] = {} def _android_client(serial: Optional[str] = None, @@ -6367,7 +6369,8 @@ def _tween_drag(start: List[int], end: List[int], steps: int = 30, button: str = "mouse_left") -> Dict[str, Any]: """Adapter: drag along an eased path from start to end.""" from je_auto_control.utils.tween_drag import tween_drag - result = tween_drag(tuple(start), tuple(end), steps=int(steps), + result = tween_drag((int(start[0]), int(start[1])), + (int(end[0]), int(end[1])), steps=int(steps), easing=easing, button=button) return {"points": result["points"]} @@ -8013,13 +8016,14 @@ def execute_action(self, action_list: Union[list, dict], @staticmethod def _unwrap_action_list(action_list: Union[list, dict]) -> list: """Normalise the ``action_list`` argument or raise on invalid input.""" - if isinstance(action_list, dict): - action_list = action_list.get("auto_control") - if action_list is None: + actions: Any = action_list + if isinstance(actions, dict): + actions = actions.get("auto_control") + if actions is None: raise AutoControlActionNullException(executor_list_error_message) - if not isinstance(action_list, list) or len(action_list) == 0: + if not isinstance(actions, list) or len(actions) == 0: raise AutoControlActionNullException(action_is_null_error_message) - return action_list + return actions def _run_one_action(self, action: list, record: Dict[str, Any], raise_on_error: bool) -> None: diff --git a/je_auto_control/utils/redaction/policies.py b/je_auto_control/utils/redaction/policies.py index f5ab15ca..b3f1d771 100644 --- a/je_auto_control/utils/redaction/policies.py +++ b/je_auto_control/utils/redaction/policies.py @@ -19,7 +19,7 @@ from __future__ import annotations from dataclasses import dataclass -from typing import List, Optional, Tuple +from typing import Optional, Sequence, Tuple # Detector tag constants (avoid magic strings in the engine + tests). @@ -50,7 +50,7 @@ class RedactionPolicy: overlay_color: Optional[Tuple[int, int, int]] = None def with_extra_regions( - self, extras: List[Tuple[int, int, int, int]]) -> "RedactionPolicy": + self, extras: Sequence[Sequence[int]]) -> "RedactionPolicy": """Return a policy with ``extras`` appended to ``regions``.""" return RedactionPolicy( detectors=tuple(self.detectors), diff --git a/test/verify/typing_contract_exempt.txt b/test/verify/typing_contract_exempt.txt index 059fd348..6a5f4756 100644 --- a/test/verify/typing_contract_exempt.txt +++ b/test/verify/typing_contract_exempt.txt @@ -14,7 +14,7 @@ # "Does not type-check" means on at least one of the three targets mypy can be # pointed at (win32, linux, darwin) — not just the one CI happens to run on. # -# Measured entries: 56 +# Measured entries: 55 je_auto_control.__main__ je_auto_control.cli @@ -38,7 +38,6 @@ je_auto_control.utils.dbus_client.session_bus je_auto_control.utils.deterministic.deterministic je_auto_control.utils.element_parse.element_parse je_auto_control.utils.element_proposal.element_proposal -je_auto_control.utils.executor.action_executor je_auto_control.utils.expect_poll.expect_poll je_auto_control.utils.feature_match.feature_match je_auto_control.utils.flow_debugger.flow_debugger From d0b49cfe8c419eb3c26357ab373326301f3834c9 Mon Sep 17 00:00:00 2001 From: JeffreyChen Date: Sat, 22 Aug 2026 21:31:19 +0800 Subject: [PATCH 17/19] Clear thirteen more, and give lastrowid one place to be checked MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit `cursor.lastrowid` is None until an INSERT has run on that cursor, and three subsystems returned `int(cursor.lastrowid)` straight out of an insert helper — work_queue, agent_memory and run_history. They now go through `sqlite_support.last_row_id`, which says what it means when the id is not there instead of handing None on as a row id. The rest of the batch is the shapes the tail keeps repeating: * Three `__exit__` methods annotated `-> bool` while every path returns False. `Literal[False]` is what they do. * Five implicit-Optional defaults (`x: str = None`). * `base64.binascii.Error` again, in the admin client — the same import side effect `totp` was catching. * `DeterministicRun._freeze_clock` multiplied `self._freeze_time` by 1e9 without checking it; its only caller guards on the same attribute, so the invariant was real but lived one frame away. --- architecture_explore.md | 34 +++++++++---------- je_auto_control/utils/admin/admin_client.py | 3 +- .../utils/agent_memory/agent_memory.py | 6 ++-- je_auto_control/utils/bulkhead/bulkhead.py | 4 +-- .../utils/critical_exit/critical_exit.py | 5 +-- .../utils/deterministic/deterministic.py | 6 ++-- .../utils/governance/credential_broker.py | 4 +-- .../utils/http_client/http_client.py | 5 ++- .../utils/project/create_project_structure.py | 10 ++++-- .../utils/run_history/history_store.py | 4 +-- .../utils/script_vars/interpolate.py | 4 +-- .../utils/soft_assert/soft_assert.py | 4 +-- je_auto_control/utils/sqlite_support.py | 20 +++++++++-- .../utils/test_record/record_test_class.py | 4 ++- .../utils/work_queue/work_queue.py | 6 ++-- test/verify/typing_contract_exempt.txt | 15 +------- 16 files changed, 76 insertions(+), 58 deletions(-) diff --git a/architecture_explore.md b/architecture_explore.md index 29200b4d..f1cb5b6e 100644 --- a/architecture_explore.md +++ b/architecture_explore.md @@ -20,7 +20,7 @@ iOS(WebDriverAgent)。核心能力是滑鼠/鍵盤控制、影像辨識、 | 指標 | 數值 | | --- | ---: | | Python 模組總數(含周邊子專案) | 1,031 | -| 程式碼總行數 | 140,867 | +| 程式碼總行數 | 140,898 | | `je_auto_control/utils/` 子套件數 | 310 | | `AC_*` 動作指令數(`known_commands()` 實測) | 773 | | 套件門面 `__all__` 公開名稱數 | 1,238 | @@ -159,7 +159,7 @@ socket server 有 8 MiB 讀取上限與 30 秒 handler timeout。 | `je_auto_control/api/core.py` | 19 | **穩定無頭 API 門面**:只暴露 `execute_action`、`execute_action_with_vars`、`generate_code`、`run_diagnostics`、`create_failure_bundle`、`failure_bundle_on_error`、`FailureBundleOptions`。mypy 型別契約以此為起點,現已擴到整包(見「設定基線」)。 | | `je_auto_control/utils/deprecation.py` | 35 | 公開 API 的一致性棄用警告。 | | `je_auto_control/utils/http_headers.py` | 45 | 入站 HTTP 標頭的共用防禦式解析。 | -| `je_auto_control/utils/sqlite_support.py` | 56 | 選用標準函式庫 `sqlite3` 的取用點:`require_sqlite3()`/`sqlite3_available()`/`SQLITE_ERRORS`。十個以 SQLite 存放狀態的子系統都經由這裡,所以 FreeBSD 這種把 `sqlite3` 另外包成 `databases/py-sqlite3` 的 Python 仍然 import 得起門面。 | +| `je_auto_control/utils/sqlite_support.py` | 70 | 選用標準函式庫 `sqlite3` 的取用點:`require_sqlite3()`/`sqlite3_available()`/`SQLITE_ERRORS`。十個以 SQLite 存放狀態的子系統都經由這裡,所以 FreeBSD 這種把 `sqlite3` 另外包成 `databases/py-sqlite3` 的 Python 仍然 import 得起門面。 | ### 5.2 wrapper 抽象層 @@ -268,7 +268,7 @@ socket server 有 8 MiB 讀取上限與 30 秒 handler timeout。 ### 5.4.1 執行引擎與腳本資產 -> 24 個套件、約 12,906 行。 +> 24 個套件、約 12,916 行。 | 模組 | 行數 | 職責 | | --- | ---: | --- | @@ -278,7 +278,7 @@ socket server 有 8 MiB 讀取上限與 30 秒 handler timeout。 | `utils/codegen/` | 157 | 由 action list 產生可執行的 pytest / python / robot 測試碼 | | `utils/dag/` | 475 | 跨主機 DAG 編排器(圖模型 + runner) | | `utils/decision_table/` | 103 | DMN 風格決策表:規則 + 命中策略,把分支外部化 | -| `utils/deterministic/` | 96 | 決定性執行控制:固定亂數種子 + 凍結時鐘 | +| `utils/deterministic/` | 98 | 決定性執行控制:固定亂數種子 + 凍結時鐘 | | `utils/executor/` | 9,081 | **核心**。`Executor` 指令分派表(773 個 `AC_*`)、參數插值、乾跑、逐步 callback;`flow_control` 提供 34 個區塊指令(迴圈/分支/try/巨集/變數) | | `utils/flow_debugger/` | 136 | action list 的單步除錯器與追蹤器 | | `utils/input_macro/` | 342 | 定時輸入事件:錄製結果的整形(`timeline`/`InputRecorder`,Windows 與 macOS 共用)、重播與宣告式輸入序列 DSL | @@ -287,25 +287,25 @@ socket server 有 8 MiB 讀取上限與 30 秒 handler timeout。 | `utils/loop_guard/` | 140 | 機械式卡死迴圈偵測(agent loop 用) | | `utils/plugin_loader/` | 85 | 掃描外部 Python 外掛目錄並註冊其 `AC_` callable | | `utils/plugin_sdk/` | 68 | 外掛 SDK:透過 entry points 發佈/載入第三方 `AC_*` 指令 | -| `utils/project/` | 182 | 專案腳手架:建立目錄結構與範本 action 檔 | +| `utils/project/` | 186 | 專案腳手架:建立目錄結構與範本 action 檔 | | `utils/recording_edit/` | 150 | 不重錄的前提下裁切/過濾/縮放已錄製的 action list | | `utils/saga/` | 93 | Saga 協調器:失敗時以 LIFO 補償動作回滾 | | `utils/script_vars/` | 190 | 執行期變數作用域與 `${var}` / `${secrets.*}` 插值 | | `utils/skill_library/` | 116 | 具名可重用 action 序列(skill)的持久化倉庫 | | `utils/state_machine/` | 181 | 宣告式有限狀態機驅動 action JSON | | `utils/stubs/` | 236 | 為 `AC_*` 指令面產生型別 stub | -| `utils/test_record/` | 64 | 全域測試紀錄單例,記錄每個動作的參數與例外 | -| `utils/work_queue/` | 180 | 交易式工作佇列(dispatcher/performer),支撐大量批次執行 | +| `utils/test_record/` | 66 | 全域測試紀錄單例,記錄每個動作的參數與例外 | +| `utils/work_queue/` | 182 | 交易式工作佇列(dispatcher/performer),支撐大量批次執行 | ### 5.4.2 框架基礎設施 -> 14 個套件、約 2,650 行。 +> 14 個套件、約 2,651 行。 | 模組 | 行數 | 職責 | | --- | ---: | --- | | `utils/callback/` | 200 | Observer 模式:`callback_executor` 以字串名觸發功能,執行後呼叫回呼 | | `utils/config_bundle/` | 400 | 使用者設定的單檔匯出/匯入 | -| `utils/critical_exit/` | 97 | 監看緊急停止鍵的守護執行緒,用於中止失控腳本 | +| `utils/critical_exit/` | 98 | 監看緊急停止鍵的守護執行緒,用於中止失控腳本 | | `utils/diagnostics/` | 322 | 跨子系統的「一切正常嗎」健檢,附 `python -m` 進入點 | | `utils/dbus_client/` | 680 | 只用標準函式庫的 D-Bus session bus 客戶端。原本在 `linux_wayland/` 為 portal 交握而寫,AT-SPI 無障礙後端成為第二個使用者後搬到這裡(`utils/` 在分層上在各 OS 套件之上) | | `utils/exception/` | 210 | **例外階層根**。所有錯誤繼承 `AutoControlException`,加上集中式錯誤訊息字串(`exception_tags`) | @@ -490,13 +490,13 @@ socket server 有 8 MiB 讀取上限與 30 秒 handler timeout。 ### 5.4.9 AI / Agent / LLM -> 13 個套件、約 20,641 行。 +> 13 個套件、約 20,643 行。 | 模組 | 行數 | 職責 | | --- | ---: | --- | | `utils/a2a/` | 92 | A2A(agent-to-agent)agent card 產生 | | `utils/agent/` | 1,250 | 閉環 Computer-Use Agent 主迴圈 + Anthropic/OpenAI/Computer-Use 三後端 | -| `utils/agent_memory/` | 151 | agent 的持久化情節記憶(goal → trajectory → outcome) | +| `utils/agent_memory/` | 153 | agent 的持久化情節記憶(goal → trajectory → outcome) | | `utils/agent_replay/` | 63 | 可攜的 agent 軌跡追蹤(記錄 observation→action 並重播) | | `utils/agent_trace/` | 129 | agent 可觀測性:OpenTelemetry GenAI 慣例的 LLM span | | `utils/cost_telemetry/` | 292 | 每次呼叫的 LLM 成本遙測:token 數 + 估算美金 | @@ -510,11 +510,11 @@ socket server 有 8 MiB 讀取上限與 30 秒 handler timeout。 ### 5.4.10 遠端桌面與 USB -> 6 個套件、約 17,901 行。 +> 6 個套件、約 17,902 行。 | 模組 | 行數 | 職責 | | --- | ---: | --- | -| `utils/admin/` | 327 | 多主機管理主控台:平行輪詢 N 個 AutoControl REST 端點 | +| `utils/admin/` | 328 | 多主機管理主控台:平行輪詢 N 個 AutoControl REST 端點 | | `utils/config_sync/` | 245 | 透過訊令伺服器做跨機器設定同步 | | `utils/device_matrix/` | 138 | 行動裝置矩陣:同一 action list 於多台裝置平行執行 | | `utils/remote_desktop/` | 11,990 | **遠端桌面子系統**(56 檔/11.7K LOC):TCP/WebSocket/WebRTC 三條傳輸路徑、主機與檢視端、訊令伺服器、TURN/中繼、多檢視者、錄影、信任清單、TOTP、稽核鏈 | @@ -523,7 +523,7 @@ socket server 有 8 MiB 讀取上限與 30 秒 handler timeout。 ### 5.4.11 伺服器、網路協定與外部整合 -> 24 個套件、約 5,919 行。 +> 24 個套件、約 5,922 行。 | 模組 | 行數 | 職責 | | --- | ---: | --- | @@ -533,7 +533,7 @@ socket server 有 8 MiB 讀取上限與 30 秒 handler timeout。 | `utils/email_send/` | 116 | SMTP 寄信(email 觸發器的發送端搭檔) | | `utils/events/` | 82 | 對外 CloudEvents 發送(執行生命週期事件) | | `utils/http_cassette/` | 110 | 錄製/重播 HTTP 互動,做離線決定性 API 測試 | -| `utils/http_client/` | 132 | 零依賴 HTTP(S) 用戶端,供 action 步驟呼叫 API | +| `utils/http_client/` | 135 | 零依賴 HTTP(S) 用戶端,供 action 步驟呼叫 API | | `utils/http_conditional/` | 87 | 條件式 HTTP 請求與快取驗證器 | | `utils/http_content/` | 103 | HTTP 內容協商與回應解壓縮 | | `utils/http_problem/` | 116 | RFC 9457 problem+json 解析 | @@ -1044,6 +1044,6 @@ socket 預設綁 `127.0.0.1`;資源一律用 `with`。 | `osx/` | 17 | 915 | | `autocontrol-lsp/` | 8 | 744 | | `utils/hotkey/` | 7 | 727 | -| 其餘模組(約 286 個 `utils/` 子套件 + `android/`/`ios/`/周邊小工具) | 673 | 47,550 | -| **總計** | **1,025** | **140,802** | +| 其餘模組(約 286 個 `utils/` 子套件 + `android/`/`ios/`/周邊小工具) | 673 | 47,581 | +| **總計** | **1,025** | **140,833** | diff --git a/je_auto_control/utils/admin/admin_client.py b/je_auto_control/utils/admin/admin_client.py index 493c3fc5..c1701911 100644 --- a/je_auto_control/utils/admin/admin_client.py +++ b/je_auto_control/utils/admin/admin_client.py @@ -113,6 +113,7 @@ def fetch_thumbnails(self, *, labels: Optional[List[str]] = None, scales the resulting image down to a thumbnail tile. """ import base64 + import binascii targets = self._resolve_targets(labels) if not targets: return {} @@ -132,7 +133,7 @@ def grab(host: AdminHost) -> tuple: return host.label, None try: return host.label, base64.b64decode(data) - except (ValueError, base64.binascii.Error): + except (ValueError, binascii.Error): return host.label, None with ThreadPoolExecutor(max_workers=self._max_parallel) as pool: diff --git a/je_auto_control/utils/agent_memory/agent_memory.py b/je_auto_control/utils/agent_memory/agent_memory.py index 15cd44c8..1acc4686 100644 --- a/je_auto_control/utils/agent_memory/agent_memory.py +++ b/je_auto_control/utils/agent_memory/agent_memory.py @@ -20,7 +20,9 @@ from dataclasses import dataclass, field from typing import TYPE_CHECKING, Any, Dict, List, Optional -from je_auto_control.utils.sqlite_support import require_sqlite3 +from je_auto_control.utils.sqlite_support import ( + last_row_id, require_sqlite3, +) if TYPE_CHECKING: # reason: sqlite3 types are named only in annotations import sqlite3 @@ -86,7 +88,7 @@ def remember(self, goal: str, *, steps: Optional[List[Any]] = None, "VALUES (?, ?, ?, ?, ?)", (str(goal), json.dumps(steps or []), str(outcome), json.dumps(list(tags or [])), time.time())) - return int(cur.lastrowid) + return last_row_id(cur) def get(self, episode_id: int) -> Optional[Episode]: """Return an episode by id or ``None``.""" diff --git a/je_auto_control/utils/bulkhead/bulkhead.py b/je_auto_control/utils/bulkhead/bulkhead.py index 2772ed9e..f6afb7a2 100644 --- a/je_auto_control/utils/bulkhead/bulkhead.py +++ b/je_auto_control/utils/bulkhead/bulkhead.py @@ -14,7 +14,7 @@ import threading import time from email.utils import parsedate_to_datetime -from typing import Any, Callable, Dict, Mapping, Optional +from typing import Any, Callable, Dict, Literal, Mapping, Optional from je_auto_control.utils.exception.exceptions import AutoControlException @@ -59,7 +59,7 @@ def __enter__(self) -> "Bulkhead": raise BulkheadFullError(f"{self.name} is full ({self._max})") return self - def __exit__(self, *_exc: Any) -> bool: + def __exit__(self, *_exc: Any) -> Literal[False]: self.release() return False diff --git a/je_auto_control/utils/critical_exit/critical_exit.py b/je_auto_control/utils/critical_exit/critical_exit.py index 2f11775c..3eef2b40 100644 --- a/je_auto_control/utils/critical_exit/critical_exit.py +++ b/je_auto_control/utils/critical_exit/critical_exit.py @@ -1,6 +1,6 @@ import _thread from threading import Event, Thread -from typing import Union +from typing import Optional, Union from je_auto_control.utils.logging.logging_instance import autocontrol_logger from je_auto_control.wrapper.auto_control_keyboard import _resolve_keycode @@ -33,7 +33,8 @@ def __init__(self, default_daemon: bool = True): self._exit_check_key: int = _resolve_keycode("f7") self._stop_event = Event() - def set_critical_key(self, keycode: Union[int, str] = None) -> None: + def set_critical_key(self, + keycode: Optional[Union[int, str]] = None) -> None: """ 設定退出鍵 Set critical exit key diff --git a/je_auto_control/utils/deterministic/deterministic.py b/je_auto_control/utils/deterministic/deterministic.py index a3f119f5..c5f54078 100644 --- a/je_auto_control/utils/deterministic/deterministic.py +++ b/je_auto_control/utils/deterministic/deterministic.py @@ -21,7 +21,7 @@ Imports no ``PySide6``. """ import random -from typing import Any, Dict, Optional +from typing import Any, Dict, Literal, Optional from unittest import mock _UNSET = object() @@ -68,6 +68,8 @@ def manifest(self) -> Dict[str, Any]: def _freeze_clock(self) -> None: instant = self._freeze_time + if instant is None: + return time_patch = mock.patch("time.time", return_value=instant) ns_patch = mock.patch("time.time_ns", return_value=int(instant * 1e9)) for patch in (time_patch, ns_patch): @@ -81,7 +83,7 @@ def __enter__(self) -> "DeterministicRun": self._freeze_clock() return self - def __exit__(self, *exc: Any) -> bool: + def __exit__(self, *exc: Any) -> Literal[False]: while self._patches: self._patches.pop().stop() if self._rng_state is not _UNSET: diff --git a/je_auto_control/utils/governance/credential_broker.py b/je_auto_control/utils/governance/credential_broker.py index 4325a887..e1a0a66a 100644 --- a/je_auto_control/utils/governance/credential_broker.py +++ b/je_auto_control/utils/governance/credential_broker.py @@ -19,7 +19,7 @@ """ import secrets import time -from typing import Callable, Dict, List, Optional +from typing import Any, Callable, Dict, List, Optional class CredentialBrokerError(RuntimeError): @@ -35,7 +35,7 @@ def __init__(self, """``resolver(name)`` returns the secret value; ``clock`` returns now.""" self._resolver = resolver self._clock = clock - self._leases: Dict[str, Dict[str, object]] = {} + self._leases: Dict[str, Dict[str, Any]] = {} def set_resolver(self, resolver: Callable[[str], Optional[str]]) -> None: """Configure the function that maps a secret name to its value.""" diff --git a/je_auto_control/utils/http_client/http_client.py b/je_auto_control/utils/http_client/http_client.py index 74d8887f..d81e2494 100644 --- a/je_auto_control/utils/http_client/http_client.py +++ b/je_auto_control/utils/http_client/http_client.py @@ -64,7 +64,10 @@ def _try_json(text: str) -> Any: def _read_response(response: Any) -> Dict[str, Any]: - status = int(getattr(response, "status", None) or getattr(response, "code", 0)) + raw_status: Any = getattr(response, "status", None) + if raw_status is None: + raw_status = getattr(response, "code", 0) + status = int(raw_status) text = response.read().decode("utf-8", errors="replace") raw_headers = getattr(response, "headers", None) headers = dict(raw_headers.items()) if raw_headers else {} diff --git a/je_auto_control/utils/project/create_project_structure.py b/je_auto_control/utils/project/create_project_structure.py index 59e9bcc3..10dec80e 100644 --- a/je_auto_control/utils/project/create_project_structure.py +++ b/je_auto_control/utils/project/create_project_structure.py @@ -1,6 +1,8 @@ from os import getcwd from pathlib import Path from threading import Lock +from typing import Optional + from je_auto_control.utils.json.json_file import write_action_json from je_auto_control.utils.logging.logging_instance import autocontrol_logger @@ -37,7 +39,8 @@ def _write_file(file_path: Path, content: str) -> None: file.write(content) -def create_template(parent_name: str, project_path: str = None) -> None: +def create_template(parent_name: str, + project_path: Optional[str] = None) -> None: """ Create template files in keyword and executor directories. 在 keyword 與 executor 目錄中建立範例模板檔案 @@ -74,7 +77,8 @@ def create_template(parent_name: str, project_path: str = None) -> None: ) -def create_project_dir(project_path: str = None, parent_name: str = "AutoControl") -> None: +def create_project_dir(project_path: Optional[str] = None, + parent_name: str = "AutoControl") -> None: """ Create project directory structure and templates. 建立專案目錄結構並生成範例模板檔案 @@ -92,4 +96,4 @@ def create_project_dir(project_path: str = None, parent_name: str = "AutoControl create_dir(str(Path(project_path) / parent_name / "executor")) # 建立範例模板檔案 Create template files - create_template(parent_name, project_path) \ No newline at end of file + create_template(parent_name, project_path) diff --git a/je_auto_control/utils/run_history/history_store.py b/je_auto_control/utils/run_history/history_store.py index 78f814eb..c58b499c 100644 --- a/je_auto_control/utils/run_history/history_store.py +++ b/je_auto_control/utils/run_history/history_store.py @@ -14,7 +14,7 @@ from je_auto_control.utils.logging.logging_instance import autocontrol_logger from je_auto_control.utils.sqlite_support import ( - SQLITE_ERRORS, require_sqlite3, + SQLITE_ERRORS, last_row_id, require_sqlite3, ) if TYPE_CHECKING: # reason: sqlite3 types are named only in annotations @@ -155,7 +155,7 @@ def start_run(self, source_type: str, source_id: str, " started_at, status) VALUES (?, ?, ?, ?, ?)", (source_type, source_id, script_path, ts, STATUS_RUNNING), ) - return int(cursor.lastrowid) + return last_row_id(cursor) def finish_run(self, run_id: int, status: str, error_text: Optional[str] = None, diff --git a/je_auto_control/utils/script_vars/interpolate.py b/je_auto_control/utils/script_vars/interpolate.py index 270d21f3..6d8c3dd3 100644 --- a/je_auto_control/utils/script_vars/interpolate.py +++ b/je_auto_control/utils/script_vars/interpolate.py @@ -14,7 +14,7 @@ import json import re from pathlib import Path -from typing import Any, Mapping, MutableMapping +from typing import Any, Mapping, MutableMapping, Optional # Bounded character class with a single quantifier — avoids the nested # alternation that ReDoS scanners (semgrep regex_dos) flag on @@ -104,7 +104,7 @@ def _lookup_secret(secret_name: str) -> str: def load_vars_from_json(path: str, - into: MutableMapping[str, Any] = None + into: Optional[MutableMapping[str, Any]] = None ) -> MutableMapping[str, Any]: """Load a flat JSON object as a variable bag.""" with open(Path(path), encoding="utf-8") as file: diff --git a/je_auto_control/utils/soft_assert/soft_assert.py b/je_auto_control/utils/soft_assert/soft_assert.py index cbf58a0b..24510c8d 100644 --- a/je_auto_control/utils/soft_assert/soft_assert.py +++ b/je_auto_control/utils/soft_assert/soft_assert.py @@ -8,7 +8,7 @@ Pure-stdlib context manager; imports no ``PySide6``. """ -from typing import Any, List +from typing import Any, List, Literal from je_auto_control.utils.exception.exceptions import AutoControlActionException @@ -52,7 +52,7 @@ def assert_all(self) -> None: def __enter__(self) -> "SoftAssertions": return self - def __exit__(self, exc_type, _exc, _tb) -> bool: + def __exit__(self, exc_type, _exc, _tb) -> Literal[False]: if exc_type is None and self._raise_on_exit: self.assert_all() return False diff --git a/je_auto_control/utils/sqlite_support.py b/je_auto_control/utils/sqlite_support.py index 9dec7cdc..90c27979 100644 --- a/je_auto_control/utils/sqlite_support.py +++ b/je_auto_control/utils/sqlite_support.py @@ -9,10 +9,11 @@ keyboard included, neither of which touches a database. Going through here instead defers the failure to the first call that actually opens one. """ -from typing import Tuple, Type +from typing import Any, Tuple, Type -from je_auto_control.utils.exception.exceptions import \ - AutoControlUnsupportedOperationException +from je_auto_control.utils.exception.exceptions import ( + AutoControlException, AutoControlUnsupportedOperationException, +) try: import sqlite3 as _sqlite3 @@ -38,6 +39,19 @@ ) +def last_row_id(cursor: Any) -> int: + """Return the id of the row a just-executed INSERT created. + + ``Cursor.lastrowid`` is ``None`` until an INSERT has run on that + cursor, so every caller that returns it has to say what it means when + it is not there rather than hand ``None`` on as an id. + """ + row_id = cursor.lastrowid + if row_id is None: + raise AutoControlException("INSERT did not report a row id") + return int(row_id) + + def sqlite3_available() -> bool: """Whether this interpreter can open SQLite databases.""" return _sqlite3 is not None diff --git a/je_auto_control/utils/test_record/record_test_class.py b/je_auto_control/utils/test_record/record_test_class.py index 36169ce3..aa26afd9 100644 --- a/je_auto_control/utils/test_record/record_test_class.py +++ b/je_auto_control/utils/test_record/record_test_class.py @@ -1,4 +1,5 @@ import datetime +from typing import Optional from je_auto_control.utils.logging.logging_instance import autocontrol_logger @@ -42,7 +43,8 @@ def set_record_enable(self, set_enable: bool = True) -> None: test_record_instance = TestRecord() -def record_action_to_list(function_name: str, local_param, program_exception: str = None) -> None: +def record_action_to_list(function_name: str, local_param, + program_exception: Optional[str] = None) -> None: """ 將動作紀錄加入清單 Record action to list diff --git a/je_auto_control/utils/work_queue/work_queue.py b/je_auto_control/utils/work_queue/work_queue.py index 9c13d58d..33b1b4c9 100644 --- a/je_auto_control/utils/work_queue/work_queue.py +++ b/je_auto_control/utils/work_queue/work_queue.py @@ -21,7 +21,9 @@ from typing import TYPE_CHECKING, Any, Dict, List, Optional from je_auto_control.utils.exception.exceptions import AutoControlException -from je_auto_control.utils.sqlite_support import require_sqlite3 +from je_auto_control.utils.sqlite_support import ( + last_row_id, require_sqlite3, +) if TYPE_CHECKING: # reason: sqlite3 types are named only in annotations import sqlite3 @@ -83,7 +85,7 @@ def add(self, data: Dict[str, Any], *, reference: Optional[str] = None, "updated) VALUES (?, ?, ?, ?, ?)", (self._name, reference or "", json.dumps(data), STATUS_NEW, time.time())) - return int(cur.lastrowid) + return last_row_id(cur) def _has_pending(self, conn: "sqlite3.Connection", reference: str) -> bool: row = conn.execute( diff --git a/test/verify/typing_contract_exempt.txt b/test/verify/typing_contract_exempt.txt index 6a5f4756..3821e187 100644 --- a/test/verify/typing_contract_exempt.txt +++ b/test/verify/typing_contract_exempt.txt @@ -14,36 +14,29 @@ # "Does not type-check" means on at least one of the three targets mypy can be # pointed at (win32, linux, darwin) — not just the one CI happens to run on. # -# Measured entries: 55 +# Measured entries: 42 je_auto_control.__main__ je_auto_control.cli je_auto_control.gui.remote_desktop.webrtc_panel je_auto_control.utils.act_modes.act_modes -je_auto_control.utils.admin.admin_client -je_auto_control.utils.agent_memory.agent_memory je_auto_control.utils.annotate.annotate -je_auto_control.utils.bulkhead.bulkhead je_auto_control.utils.change_localize.change_localize je_auto_control.utils.codegen.codegen je_auto_control.utils.color_match.color_match je_auto_control.utils.color_stats.color_stats je_auto_control.utils.config_sync.client -je_auto_control.utils.critical_exit.critical_exit je_auto_control.utils.cv2_utils.screen_record je_auto_control.utils.cv2_utils.video_recording je_auto_control.utils.dag.runner je_auto_control.utils.data_quality.data_quality je_auto_control.utils.dbus_client.session_bus -je_auto_control.utils.deterministic.deterministic je_auto_control.utils.element_parse.element_parse je_auto_control.utils.element_proposal.element_proposal je_auto_control.utils.expect_poll.expect_poll je_auto_control.utils.feature_match.feature_match je_auto_control.utils.flow_debugger.flow_debugger je_auto_control.utils.generate_report.generate_xml_report -je_auto_control.utils.governance.credential_broker -je_auto_control.utils.http_client.http_client je_auto_control.utils.humanize.motion je_auto_control.utils.json_patch.json_patch je_auto_control.utils.locale_collation.locale_collation @@ -52,22 +45,16 @@ je_auto_control.utils.mouse_path.mouse_path je_auto_control.utils.ocr.backends je_auto_control.utils.preprocess.preprocess je_auto_control.utils.profiler.resource_profiler -je_auto_control.utils.project.create_project_structure je_auto_control.utils.qr.qr je_auto_control.utils.rate_limit.rate_limit je_auto_control.utils.rotated_match.rotated_match -je_auto_control.utils.run_history.history_store je_auto_control.utils.sbom.sbom je_auto_control.utils.scheduler.scheduler -je_auto_control.utils.script_vars.interpolate je_auto_control.utils.search_index.search_index je_auto_control.utils.secrets.secret_store je_auto_control.utils.socket_server.auto_control_socket_server -je_auto_control.utils.soft_assert.soft_assert -je_auto_control.utils.test_record.record_test_class je_auto_control.utils.text_normalize.text_normalize je_auto_control.utils.tls_acme.challenge je_auto_control.utils.visual_regression.compare je_auto_control.utils.watcher.watcher -je_auto_control.utils.work_queue.work_queue je_auto_control.utils.xml.xml_file.xml_file From f68e02bafc48d60df0666dbf222385cca9e13408 Mon Sep 17 00:00:00 2001 From: JeffreyChen Date: Sat, 22 Aug 2026 21:48:19 +0800 Subject: [PATCH 18/19] Empty the typing contract's exemption list but for one module MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Forty-one modules in one sweep, and the list is now a single line: `gui/remote_desktop/webrtc_panel.py`, blocked on the split its size cap already owes it (recorded in Progress.md). Several were annotations that were simply wrong about what the code returns, and fixing them is what let the callers stop working around it: * `screenshot()` said `-> List[int]` while it returns the captured BGR frame. Every caller that read `.shape` off it was reading through a lie; `screen_record` was the one mypy caught. * `visual_match._nms` was typed for `Match` and reused from `rotated_match`, `edge_match` and `color_match` with their own record types. It takes a `TypeVar` bound to a read-only Protocol of what it actually reads — position, size, score — which is also why the bound had to be properties: the records are frozen dataclasses, and a Protocol attribute demands a writable one. * `logical_frame`'s injectable grabber was typed `Callable[..., Any]` while its own docstring says "ImageGrab-shaped object", so `.grab()` on it was an error. Four fix a `None` the code could really see: `humanize.motion` and `watcher` unpacked `get_mouse_position()` (the Windows backend returns None on a locked desktop), `dag.runner` opened a node's `action_file` without checking a node had one, and `color_match` divided a `None` accumulator when handed no channels. `secret_store._require_unlocked` now returns the `(fernet, vault)` pair it checks, which removed three `type: ignore`s that existed only because the check and the use were separate statements. Two more cv2 names — `VideoWriter_fourcc` and `ORB_create` — join `MSER_create` as present at runtime but absent from the shipped stub. --- architecture_explore.md | 84 +++++++++---------- je_auto_control/__main__.py | 16 ++-- je_auto_control/cli.py | 4 +- je_auto_control/utils/act_modes/act_modes.py | 5 +- je_auto_control/utils/annotate/annotate.py | 5 +- .../utils/change_localize/change_localize.py | 2 +- je_auto_control/utils/codegen/codegen.py | 7 +- .../utils/color_match/color_match.py | 2 + .../utils/color_stats/color_stats.py | 3 +- je_auto_control/utils/config_sync/client.py | 3 +- .../utils/cv2_utils/video_recording.py | 2 +- je_auto_control/utils/dag/runner.py | 3 + .../utils/data_quality/data_quality.py | 7 +- .../utils/dbus_client/session_bus.py | 7 +- .../utils/element_parse/element_parse.py | 2 +- .../element_proposal/element_proposal.py | 2 +- .../utils/expect_poll/expect_poll.py | 4 +- .../utils/feature_match/feature_match.py | 5 +- .../utils/flow_debugger/flow_debugger.py | 10 ++- .../generate_report/generate_xml_report.py | 6 +- je_auto_control/utils/humanize/motion.py | 7 ++ .../utils/json_patch/json_patch.py | 2 +- .../locale_collation/locale_collation.py | 4 +- .../utils/monitor_layout/logical_frame.py | 4 +- .../utils/mouse_path/mouse_path.py | 4 +- .../utils/ocr/backends/__init__.py | 1 + .../utils/preprocess/preprocess.py | 4 +- .../utils/profiler/resource_profiler.py | 4 +- je_auto_control/utils/qr/qr.py | 3 +- .../utils/rate_limit/rate_limit.py | 4 +- je_auto_control/utils/sbom/sbom.py | 4 +- je_auto_control/utils/scheduler/scheduler.py | 2 +- .../utils/search_index/search_index.py | 6 +- je_auto_control/utils/secrets/secret_store.py | 23 ++--- .../auto_control_socket_server.py | 2 +- .../utils/text_normalize/text_normalize.py | 11 ++- je_auto_control/utils/tls_acme/challenge.py | 7 +- .../utils/visual_match/visual_match.py | 35 +++++++- .../utils/visual_regression/compare.py | 9 +- je_auto_control/utils/watcher/watcher.py | 6 +- .../utils/xml/xml_file/xml_file.py | 2 + .../wrapper/auto_control_screen.py | 4 +- test/verify/typing_contract_exempt.txt | 43 +--------- 43 files changed, 210 insertions(+), 160 deletions(-) diff --git a/architecture_explore.md b/architecture_explore.md index f1cb5b6e..bb4e27be 100644 --- a/architecture_explore.md +++ b/architecture_explore.md @@ -20,7 +20,7 @@ iOS(WebDriverAgent)。核心能力是滑鼠/鍵盤控制、影像辨識、 | 指標 | 數值 | | --- | ---: | | Python 模組總數(含周邊子專案) | 1,031 | -| 程式碼總行數 | 140,898 | +| 程式碼總行數 | 140,989 | | `je_auto_control/utils/` 子套件數 | 310 | | `AC_*` 動作指令數(`known_commands()` 實測) | 773 | | 套件門面 `__all__` 公開名稱數 | 1,238 | @@ -153,7 +153,7 @@ socket server 有 8 MiB 讀取上限與 30 秒 handler timeout。 | 模組 | 行數 | 職責 | | --- | ---: | --- | | `je_auto_control/__init__.py` | 1,970 | **套件門面**。集中匯入並再匯出 1,200 個公開名稱,以功能區塊註解分段(callback/exception/executor/a11y/vision/clipboard…)。 | -| `je_auto_control/__main__.py` | 70 | 舊版 argparse 進入點:`-e` 執行單檔、`-d` 執行整個目錄、`--execute_str` 執行 JSON 字串、`-c` 建立專案。 | +| `je_auto_control/__main__.py` | 74 | 舊版 argparse 進入點:`-e` 執行單檔、`-d` 執行整個目錄、`--execute_str` 執行 JSON 字串、`-c` 建立專案。 | | `je_auto_control/cli.py` | 323 | **主 CLI**(`je_auto_control` console script)。子命令:`run`(含 `--var`/`--dry-run`)、`validate`/`lint`、`list-commands`、`fmt`、`record`、`codegen`、`failure-bundle`、`list-jobs`、`start-server`、`start-rest`、`version`。所有子命令延遲匯入,確保不碰 Qt。 | | `je_auto_control/api/__init__.py` | 22 | 版本化整合進入點。 | | `je_auto_control/api/core.py` | 19 | **穩定無頭 API 門面**:只暴露 `execute_action`、`execute_action_with_vars`、`generate_code`、`run_diagnostics`、`create_failure_bundle`、`failure_bundle_on_error`、`FailureBundleOptions`。mypy 型別契約以此為起點,現已擴到整包(見「設定基線」)。 | @@ -268,19 +268,19 @@ socket server 有 8 MiB 讀取上限與 30 秒 handler timeout。 ### 5.4.1 執行引擎與腳本資產 -> 24 個套件、約 12,916 行。 +> 24 個套件、約 12,926 行。 | 模組 | 行數 | 職責 | | --- | ---: | --- | | `utils/action_lint/` | 328 | action 檔 linter 與 JSON Schema 產生器(CI 用 `python -m` 進入點) | | `utils/action_signing/` | 248 | action 檔 HMAC-SHA256 簽章與 Fernet 加密,`execute_files` 會強制驗簽 | | `utils/checkpoint/` | 120 | 流程檢查點與續跑,讓長 action list 具持久性 | -| `utils/codegen/` | 157 | 由 action list 產生可執行的 pytest / python / robot 測試碼 | -| `utils/dag/` | 475 | 跨主機 DAG 編排器(圖模型 + runner) | +| `utils/codegen/` | 158 | 由 action list 產生可執行的 pytest / python / robot 測試碼 | +| `utils/dag/` | 478 | 跨主機 DAG 編排器(圖模型 + runner) | | `utils/decision_table/` | 103 | DMN 風格決策表:規則 + 命中策略,把分支外部化 | | `utils/deterministic/` | 98 | 決定性執行控制:固定亂數種子 + 凍結時鐘 | | `utils/executor/` | 9,081 | **核心**。`Executor` 指令分派表(773 個 `AC_*`)、參數插值、乾跑、逐步 callback;`flow_control` 提供 34 個區塊指令(迴圈/分支/try/巨集/變數) | -| `utils/flow_debugger/` | 136 | action list 的單步除錯器與追蹤器 | +| `utils/flow_debugger/` | 142 | action list 的單步除錯器與追蹤器 | | `utils/input_macro/` | 342 | 定時輸入事件:錄製結果的整形(`timeline`/`InputRecorder`,Windows 與 macOS 共用)、重播與宣告式輸入序列 DSL | | `utils/json/` | 74 | action JSON 檔讀寫與正規化格式化(`fmt --check` 的後端) | | `utils/json_store/` | 61 | JSON 字典檔持久化的共用小工具(內部管線) | @@ -299,7 +299,7 @@ socket server 有 8 MiB 讀取上限與 30 秒 handler timeout。 ### 5.4.2 框架基礎設施 -> 14 個套件、約 2,651 行。 +> 14 個套件、約 2,654 行。 | 模組 | 行數 | 職責 | | --- | ---: | --- | @@ -307,7 +307,7 @@ socket server 有 8 MiB 讀取上限與 30 秒 handler timeout。 | `utils/config_bundle/` | 400 | 使用者設定的單檔匯出/匯入 | | `utils/critical_exit/` | 98 | 監看緊急停止鍵的守護執行緒,用於中止失控腳本 | | `utils/diagnostics/` | 322 | 跨子系統的「一切正常嗎」健檢,附 `python -m` 進入點 | -| `utils/dbus_client/` | 680 | 只用標準函式庫的 D-Bus session bus 客戶端。原本在 `linux_wayland/` 為 portal 交握而寫,AT-SPI 無障礙後端成為第二個使用者後搬到這裡(`utils/` 在分層上在各 OS 套件之上) | +| `utils/dbus_client/` | 683 | 只用標準函式庫的 D-Bus session bus 客戶端。原本在 `linux_wayland/` 為 portal 交握而寫,AT-SPI 無障礙後端成為第二個使用者後搬到這裡(`utils/` 在分層上在各 OS 套件之上) | | `utils/exception/` | 210 | **例外階層根**。所有錯誤繼承 `AutoControlException`,加上集中式錯誤訊息字串(`exception_tags`) | | `utils/failure_bundle/` | 187 | 可攜、已遮蔽的失敗診斷 ZIP(截圖 + 診斷 + log 尾段) | | `utils/file_process/` | 26 | 目錄檔案列舉(`execute_dir` 的後端) | @@ -320,7 +320,7 @@ socket server 有 8 MiB 讀取上限與 30 秒 handler timeout。 ### 5.4.3 排程、觸發與背景監看 -> 11 個套件、約 3,550 行。 +> 11 個套件、約 3,554 行。 | 模組 | 行數 | 職責 | | --- | ---: | --- | @@ -334,27 +334,27 @@ socket server 有 8 MiB 讀取上限與 30 秒 handler timeout。 | `utils/triggers/` | 1,152 | 事件驅動觸發引擎:影像/視窗/像素/檔案/webhook/IMAP 郵件 | | `utils/voice/` | 87 | 語音指令路由:把辨識到的語句對應到 `AC_*` action list | | `utils/watchdog/` | 173 | 背景彈窗/中斷看門狗,供無人值守自動化 | -| `utils/watcher/` | 78 | 無頭輪詢原語:滑鼠位置、像素顏色、log tail | +| `utils/watcher/` | 82 | 無頭輪詢原語:滑鼠位置、像素顏色、log tail | ### 5.4.4 輸入模擬與動作品質 -> 22 個套件、約 2,617 行。 +> 22 個套件、約 2,627 行。 | 模組 | 行數 | 職責 | | --- | ---: | --- | | `utils/act_in_view/` | 76 | 先把目標捲進視野,待其可操作後再動作 | -| `utils/act_modes/` | 67 | actionability 閘門之上的 trial/force 動作模式 | +| `utils/act_modes/` | 68 | actionability 閘門之上的 trial/force 動作模式 | | `utils/action_effect/` | 110 | 判定一個動作是否真的產生效果,並歸因到目標區域 | | `utils/action_grounding/` | 80 | 動作前的接地守衛(邊界檢查 + 吸附到元素) | | `utils/actionability/` | 166 | 動作前就緒閘門(可見 + 穩定 + 啟用 + 未被遮擋) | | `utils/ensure_state/` | 74 | 冪等地把控制項/設定帶到期望狀態 | | `utils/field_entry/` | 76 | 清空再輸入的欄位填寫慣用法(Playwright `fill`) | | `utils/gamepad/` | 311 | 虛擬遊戲手把後端(Windows ViGEmBus 驅動) | -| `utils/humanize/` | 183 | 擬人輸入:貝茲曲線滑鼠路徑 + 抖動打字節奏 | +| `utils/humanize/` | 190 | 擬人輸入:貝茲曲線滑鼠路徑 + 抖動打字節奏 | | `utils/ime_state/` | 144 | 讀取即時 IME 組字/轉換狀態,確保 CJK 輸入安全 | | `utils/key_hold/` | 107 | 按住按鍵一段時間,或以固定頻率自動重複 | | `utils/modifier_state/` | 76 | 跨一組動作按住修飾鍵,並保證安全釋放 | -| `utils/mouse_path/` | 92 | 多路徑點滑鼠手勢(沿折線移動或拖曳) | +| `utils/mouse_path/` | 94 | 多路徑點滑鼠手勢(沿折線移動或拖曳) | | `utils/mouse_relative/` | 59 | 相對位移滑鼠移動 | | `utils/postcondition/` | 138 | 宣告式的動作預期結果規格,對照畫面驗證 | | `utils/step_repair/` | 114 | 失敗/無效動作的修復策略(自我修正迴圈) | @@ -367,20 +367,20 @@ socket server 有 8 MiB 讀取上限與 30 秒 handler timeout。 ### 5.4.5 影像辨識與畫面分析 -> 37 個套件、約 5,067 行。 +> 37 個套件、約 5,105 行。 | 模組 | 行數 | 職責 | | --- | ---: | --- | -| `utils/annotate/` | 114 | 截圖標註:畫框、highlight、箭頭、標籤 | +| `utils/annotate/` | 115 | 截圖標註:畫框、highlight、箭頭、標籤 | | `utils/barcode/` | 53 | 一維條碼(EAN/UPC)解碼,解碼器可注入 | -| `utils/color_match/` | 103 | 在 HSV 通道上做顏色感知的樣板比對 | +| `utils/color_match/` | 105 | 在 HSV 通道上做顏色感知的樣板比對 | | `utils/color_region/` | 79 | 以顏色定位畫面區域(遮罩 + 連通元件) | -| `utils/color_stats/` | 95 | 區域顏色統計:平均色與主色 | +| `utils/color_stats/` | 96 | 區域顏色統計:平均色與主色 | | `utils/coordinate_space/` | 84 | 模型網格座標與實體像素之間的座標空間對映 | | `utils/cv2_utils/` | 637 | OpenCV 基礎層:擷取後端選擇(`screen_grabber`,Pillow/mss 或平台後端)、截圖、樣板比對(走 `grab_logical`,涵蓋所有螢幕)、螢幕錄影、影片錄製、連通元件、影像堆疊的取用口(`optional`,Windows arm64 沒有 wheel 時語意報錯) | | `utils/edge_lines/` | 120 | 以 Hough 轉換偵測線條/格線/分隔線 | | `utils/edge_match/` | 112 | 邊緣形狀(Chamfer/距離轉換)樣板比對 | -| `utils/feature_match/` | 129 | ORB 特徵比對:在旋轉/縮放/主題變更下定位樣板 | +| `utils/feature_match/` | 130 | ORB 特徵比對:在旋轉/縮放/主題變更下定位樣板 | | `utils/hsv_segment/` | 91 | HSV 色彩空間分割(抗光照的顏色遮罩 + blob 框) | | `utils/icon_classify/` | 113 | 從像素形狀判斷一個框是哪一類元件 | | `utils/image_dedup/` | 83 | 感知雜湊影像去重(Pillow aHash/dHash) | @@ -395,7 +395,7 @@ socket server 有 8 MiB 讀取上限與 30 秒 handler timeout。 | `utils/motion_regions/` | 73 | 兩影格間的局部變化/活動偵測(absdiff) | | `utils/perceptual_diff/` | 100 | 感知式(YIQ)影像差異,抑制反鋸齒邊緣誤報 | | `utils/preprocess/` | 185 | OCR/比對前的影像前處理(灰階、二值化、去傾斜…) | -| `utils/qr/` | 59 | 從影像或螢幕區域解碼 QR code(OpenCV) | +| `utils/qr/` | 60 | 從影像或螢幕區域解碼 QR code(OpenCV) | | `utils/rotated_match/` | 145 | 容忍旋轉與縮放的樣板比對(尺度空間 × 角度掃描) | | `utils/saliency/` | 107 | 頻譜殘差視覺顯著性:顯著圖與排序後的顯著區域 | | `utils/scale_detect/` | 84 | 偵測樣板實際渲染的顯示縮放/視覺 DPI | @@ -406,12 +406,12 @@ socket server 有 8 MiB 讀取上限與 30 秒 handler timeout。 | `utils/subpixel_match/` | 101 | 以二次曲面擬合做次像素級比對精修 | | `utils/theme_normalize/` | 92 | 主題無關的影像正規化,讓亮色樣板能配對深色模式 | | `utils/video_report/` | 133 | 影片步驟疊圖報告:把截圖加字幕串成操作導覽影片 | -| `utils/visual_match/` | 427 | 會回傳信心值的樣板比對(分數、多尺度、find-all + NMS);擷取走 `grab_logical`,命中座標已加回虛擬桌面原點,單色樣板直接拒收 | -| `utils/visual_regression/` | 221 | 桌面 GUI 的視覺回歸測試(黃金圖比對) | +| `utils/visual_match/` | 454 | 會回傳信心值的樣板比對(分數、多尺度、find-all + NMS);擷取走 `grab_logical`,命中座標已加回虛擬桌面原點,單色樣板直接拒收 | +| `utils/visual_regression/` | 226 | 桌面 GUI 的視覺回歸測試(黃金圖比對) | ### 5.4.6 OCR 與文字理解 -> 19 個套件、約 3,184 行。 +> 19 個套件、約 3,196 行。 | 模組 | 行數 | 職責 | | --- | ---: | --- | @@ -424,14 +424,14 @@ socket server 有 8 MiB 讀取上限與 30 秒 handler timeout。 | `utils/guardrail/` | 108 | 針對畫面/OCR 文字的啟發式 prompt-injection 防護 | | `utils/heading_segment/` | 69 | 判定 OCR 行是標題或內文,建出文件大綱 | | `utils/near_dup/` | 105 | 近似重複文字偵測(SimHash/MinHash) | -| `utils/ocr/` | 1,112 | OCR 引擎門面 + 三個後端(Tesseract/EasyOCR/PaddleOCR)、版面結構化與跨詞比對(`text_span`) | +| `utils/ocr/` | 1,113 | OCR 引擎門面 + 三個後端(Tesseract/EasyOCR/PaddleOCR)、版面結構化與跨詞比對(`text_span`) | | `utils/pii_text/` | 98 | 自由文字中的 PII 偵測與遮蔽(email/電話/SSN/卡號/IP/IBAN) | | `utils/readability/` | 137 | 可讀性評分(Flesch、Flesch-Kincaid、Gunning Fog、SMOG、ARI) | | `utils/reading_flow/` | 119 | 以遞迴 XY-cut 推導欄位感知的閱讀順序 | -| `utils/search_index/` | 140 | 記憶體內 BM25/TF-IDF 全文檢索 | +| `utils/search_index/` | 142 | 記憶體內 BM25/TF-IDF 全文檢索 | | `utils/text_blocks/` | 88 | 把 OCR 行組成段落與項目符號/編號清單 | | `utils/text_diff/` | 148 | unified diff 產生、套用與三方合併 | -| `utils/text_normalize/` | 63 | Unicode 正規化與 slug 產生 | +| `utils/text_normalize/` | 72 | Unicode 正規化與 slug 產生 | | `utils/text_regions/` | 161 | 免模型的畫面文字區域偵測(MSER):區域與行 | | `utils/text_similarity/` | 165 | 字串距離度量(文字比對用) | @@ -510,12 +510,12 @@ socket server 有 8 MiB 讀取上限與 30 秒 handler timeout。 ### 5.4.10 遠端桌面與 USB -> 6 個套件、約 17,902 行。 +> 6 個套件、約 17,903 行。 | 模組 | 行數 | 職責 | | --- | ---: | --- | | `utils/admin/` | 328 | 多主機管理主控台:平行輪詢 N 個 AutoControl REST 端點 | -| `utils/config_sync/` | 245 | 透過訊令伺服器做跨機器設定同步 | +| `utils/config_sync/` | 246 | 透過訊令伺服器做跨機器設定同步 | | `utils/device_matrix/` | 138 | 行動裝置矩陣:同一 action list 於多台裝置平行執行 | | `utils/remote_desktop/` | 11,990 | **遠端桌面子系統**(56 檔/11.7K LOC):TCP/WebSocket/WebRTC 三條傳輸路徑、主機與檢視端、訊令伺服器、TURN/中繼、多檢視者、錄影、信任清單、TOTP、稽核鏈 | | `utils/usb/` | 4,281 | 跨平台 USB 列舉/熱插拔/裝置直通(WinUSB、IOKit、libusb 後端 + ACL + WebRTC DataChannel 通道) | @@ -523,7 +523,7 @@ socket server 有 8 MiB 讀取上限與 30 秒 handler timeout。 ### 5.4.11 伺服器、網路協定與外部整合 -> 24 個套件、約 5,922 行。 +> 24 個套件、約 5,923 行。 | 模組 | 行數 | 職責 | | --- | ---: | --- | @@ -548,7 +548,7 @@ socket server 有 8 MiB 讀取上限與 30 秒 handler timeout。 | `utils/rest_api/` | 1,751 | 純標準庫 REST 前端:路由、Bearer 驗證、限流、Prometheus 指標、OpenAPI 3.1 產生 | | `utils/socket_server/` | 131 | 執行 action JSON 的執行緒式 TCP 指令伺服器(預設綁 127.0.0.1) | | `utils/sse_client/` | 112 | Server-Sent Events 用戶端解析 | -| `utils/tls_acme/` | 447 | TLS 自動化:HTTP-01 挑戰伺服器、金鑰/CSR、自動續期 | +| `utils/tls_acme/` | 448 | TLS 自動化:HTTP-01 挑戰伺服器、金鑰/CSR、自動續期 | | `utils/url_canon/` | 117 | RFC 3986 URL 正規化與查詢字串工具 | | `utils/webrunner_bridge/` | 161 | 把 action JSON 橋接到 WebRunner(`je_web_runner`) | @@ -569,14 +569,14 @@ socket server 有 8 MiB 讀取上限與 30 秒 handler timeout。 | `utils/failure_signature/` | 74 | 把錯誤訊息正規化成穩定的 SHA-256 失敗簽章並分群 | | `utils/flake_cluster/` | 103 | 以共同失敗 Jaccard 相似度為易碎測試分群 | | `utils/flakiness/` | 150 | 以執行歷史分析不穩定測試 | -| `utils/generate_report/` | 310 | HTML/JSON/XML 三種報表產生器(Template Method) | +| `utils/generate_report/` | 308 | HTML/JSON/XML 三種報表產生器(Template Method) | | `utils/media_assert/` | 233 | 媒體斷言:音訊活動與影片動態檢查 | | `utils/observability/` | 668 | Prometheus 格式指標 + OpenTelemetry 相容 trace + `/metrics` 匯出伺服器 | | `utils/otlp_export/` | 81 | OTLP/JSON span 匯出 | | `utils/percentiles/` | 103 | 可合併的串流延遲摘要與精確百分位數 | | `utils/process_doc/` | 85 | 由錄製的 action list 產生逐步 SOP 文件 | | `utils/process_mining/` | 110 | 流程探勘:從動作日誌挖掘可自動化的候選 | -| `utils/profiler/` | 422 | 逐動作效能剖析器 + 資源剖析器 | +| `utils/profiler/` | 424 | 逐動作效能剖析器 + 資源剖析器 | | `utils/quarantine/` | 190 | 易碎測試隔離區,讓套件執行器跳過已知不穩定案例 | | `utils/run_diff/` | 123 | 兩次執行軌跡的差異(LCS 對齊:新增/移除/狀態翻轉/退化) | | `utils/run_history/` | 377 | 執行歷史儲存與產出物管理 | @@ -595,7 +595,7 @@ socket server 有 8 MiB 讀取上限與 30 秒 handler timeout。 ### 5.4.13 資料來源、結構驗證與 i18n -> 24 個套件、約 3,892 行。 +> 24 個套件、約 3,895 行。 | 模組 | 行數 | 職責 | | --- | ---: | --- | @@ -603,7 +603,7 @@ socket server 有 8 MiB 讀取上限與 30 秒 handler timeout。 | `utils/config_schema/` | 109 | 型別化設定結構驗證 | | `utils/data_drift/` | 125 | 分布漂移偵測 | | `utils/data_profile/` | 121 | 資料剖析與結構推斷 | -| `utils/data_quality/` | 185 | 資料品質:列結構驗證、欄位擷取、遮蔽 | +| `utils/data_quality/` | 186 | 資料品質:列結構驗證、欄位擷取、遮蔽 | | `utils/data_source/` | 182 | 資料驅動執行:從 CSV/JSON/SQLite/Excel 載入資料列 | | `utils/dataset_diff/` | 89 | 表格資料列差異比對(CDC 風格) | | `utils/gettext_catalog/` | 296 | GNU gettext 目錄 I/O(解析 .po、編譯/讀取 .mo、訊息查詢) | @@ -622,11 +622,11 @@ socket server 有 8 MiB 讀取上限與 30 秒 handler timeout。 | `utils/schema_compat/` | 162 | JSON Schema 相容性分級 | | `utils/sql/` | 78 | 對 SQLite 的臨時唯讀 SQL 查詢 | | `utils/test_data/` | 205 | 帶種子的合成測試資料產生(純標準庫) | -| `utils/xml/` | 250 | XML 檔讀寫與結構變更(`defusedxml`) | +| `utils/xml/` | 252 | XML 檔讀寫與結構變更(`defusedxml`) | ### 5.4.14 安全、機密與合規 -> 13 個套件、約 2,289 行。 +> 13 個套件、約 2,294 行。 | 模組 | 行數 | 職責 | | --- | ---: | --- | @@ -637,9 +637,9 @@ socket server 有 8 MiB 讀取上限與 30 秒 handler timeout。 | `utils/provenance/` | 104 | SLSA 建置來源證明(in-toto v1) | | `utils/rbac/` | 272 | 角色型存取控制與逐使用者稽核歸因 | | `utils/redaction/` | 467 | 截圖遮蔽層:規則偵測 + 政策 + 協調器(上傳 VLM 前先遮) | -| `utils/sbom/` | 108 | SBOM(CycloneDX)產生 | +| `utils/sbom/` | 110 | SBOM(CycloneDX)產生 | | `utils/secret_ref/` | 126 | URI scheme 形式的值參照解析 | -| `utils/secrets/` | 269 | 加密機密儲存庫,供 `${secrets.NAME}` 解析 | +| `utils/secrets/` | 272 | 加密機密儲存庫,供 `${secrets.NAME}` 解析 | | `utils/secrets_scan/` | 98 | 掃描 action JSON/資料中應入庫卻硬編碼的機密 | | `utils/vex/` | 130 | OpenVEX 陳述撰寫與漏洞分類處置 | | `utils/vuln_scan/` | 188 | 以 OSV 比對 SBOM 元件的漏洞(純標準庫) | @@ -1029,7 +1029,7 @@ socket 預設綁 `127.0.0.1`;資源一律用 `with`。 | `utils/remote_desktop/` | 56 | 11,990 | | `utils/executor/` | 6 | 9,081 | | `utils/usb/` | 17 | 4,281 | -| `je_auto_control/`(頂層 3 檔) | 3 | 2,363 | +| `je_auto_control/`(頂層 3 檔) | 3 | 2,367 | | `utils/accessibility/` | 13 | 2,835 | | `wrapper/` | 19 | 3,293 | | `windows/` | 23 | 1,906 | @@ -1038,12 +1038,12 @@ socket 預設綁 `127.0.0.1`;資源一律用 `with`。 | `linux_with_x11/` | 19 | 1,236 | | `linux_wayland/` | 17 | 2,870 | | `utils/triggers/` | 4 | 1,152 | -| `utils/ocr/` | 9 | 1,112 | +| `utils/ocr/` | 9 | 1,113 | | `utils/usbip/` | 5 | 920 | | `utils/assertion/` | 3 | 863 | | `osx/` | 17 | 915 | | `autocontrol-lsp/` | 8 | 744 | | `utils/hotkey/` | 7 | 727 | -| 其餘模組(約 286 個 `utils/` 子套件 + `android/`/`ios/`/周邊小工具) | 673 | 47,581 | -| **總計** | **1,025** | **140,833** | +| 其餘模組(約 286 個 `utils/` 子套件 + `android/`/`ios/`/周邊小工具) | 673 | 47,667 | +| **總計** | **1,025** | **140,924** | diff --git a/je_auto_control/__main__.py b/je_auto_control/__main__.py index 0f6a97f3..d25c2649 100644 --- a/je_auto_control/__main__.py +++ b/je_auto_control/__main__.py @@ -55,12 +55,16 @@ def preprocess_read_str_execute_action(execute_str: str): "--execute_str", type=str, help="execute json str" ) - args = parser.parse_args() - args = vars(args) - for key, value in args.items(): - if value is not None: - argparse_event_dict.get(key)(value) - if all(value is None for value in args.values()): + parsed = vars(parser.parse_args()) + for key, value in parsed.items(): + if value is None: + continue + handler = argparse_event_dict.get(key) + if handler is None: + raise AutoControlArgparseException( + argparse_get_wrong_data_error_message) + handler(value) + if all(value is None for value in parsed.values()): raise AutoControlArgparseException(argparse_get_wrong_data_error_message) except AutoControlArgparseException as error: autocontrol_logger.error("argparse failure: %r", error) diff --git a/je_auto_control/cli.py b/je_auto_control/cli.py index ff46edca..585fc38c 100644 --- a/je_auto_control/cli.py +++ b/je_auto_control/cli.py @@ -26,7 +26,7 @@ import sys import threading import time -from typing import Dict, List, Optional, Sequence +from typing import Callable, Dict, List, Optional, Sequence from je_auto_control.utils.exception.exceptions import ( AutoControlActionException, @@ -215,7 +215,7 @@ def cmd_start_rest(args: argparse.Namespace) -> int: return 0 -def _run_until_signal(shutdown: callable) -> None: +def _run_until_signal(shutdown: Callable[[], None]) -> None: stopping = {"flag": False} def _handler(_signum, _frame): diff --git a/je_auto_control/utils/act_modes/act_modes.py b/je_auto_control/utils/act_modes/act_modes.py index ed370fb9..84a46ed7 100644 --- a/je_auto_control/utils/act_modes/act_modes.py +++ b/je_auto_control/utils/act_modes/act_modes.py @@ -58,6 +58,7 @@ def act_with_mode(action: Callable[[List[int]], Any], point = list(report.point) if report.point is not None else None base = {"mode": mode, "actionable": report.actionable, "reason": report.reason, "point": point} - if mode == "trial" or not report.actionable: + target = report.point + if mode == "trial" or not report.actionable or target is None: return {**base, "acted": False, "result": None} - return {**base, "acted": True, "result": action(report.point)} + return {**base, "acted": True, "result": action(target)} diff --git a/je_auto_control/utils/annotate/annotate.py b/je_auto_control/utils/annotate/annotate.py index 615c6577..e2e9eb46 100644 --- a/je_auto_control/utils/annotate/annotate.py +++ b/je_auto_control/utils/annotate/annotate.py @@ -77,7 +77,8 @@ def _draw_arrow(draw: ImageDraw.ImageDraw, ann: Dict[str, Any]) -> None: def _draw_text(draw: ImageDraw.ImageDraw, ann: Dict[str, Any]) -> None: - pos = tuple(int(v) for v in ann["position"]) + raw = ann["position"] + pos = (float(raw[0]), float(raw[1])) draw.text(pos, str(ann.get("text", "")), fill=_color(ann.get("color"))) @@ -101,7 +102,7 @@ def annotate_screenshot(source: ImageSource, draw = ImageDraw.Draw(base) dispatch = {"box": _draw_box, "arrow": _draw_arrow, "text": _draw_text} for ann in annotations: - handler = dispatch.get(ann.get("type")) + handler = dispatch.get(str(ann.get("type", ""))) if handler is not None: handler(draw, ann) out = Path(output_path) diff --git a/je_auto_control/utils/change_localize/change_localize.py b/je_auto_control/utils/change_localize/change_localize.py index efb38cc6..c92302a6 100644 --- a/je_auto_control/utils/change_localize/change_localize.py +++ b/je_auto_control/utils/change_localize/change_localize.py @@ -35,7 +35,7 @@ def rank_changes(scored_boxes: Sequence[Any], *, is ``True`` when the score is at or above ``threshold``. """ limit = float(threshold) - result = [ + result: List[Dict[str, Any]] = [ {"box": [int(value) for value in box], "score": round(float(score), 4), "changed": float(score) >= limit} diff --git a/je_auto_control/utils/codegen/codegen.py b/je_auto_control/utils/codegen/codegen.py index e5986197..679bf149 100644 --- a/je_auto_control/utils/codegen/codegen.py +++ b/je_auto_control/utils/codegen/codegen.py @@ -42,13 +42,14 @@ def _action_to_call(action: Sequence, event_dict: dict, public: set) -> str: name = action[0] params = action[1] if len(action) == 2 else None func = event_dict.get(name) - direct = func is not None and getattr(func, "__name__", "") in public + public_name = getattr(func, "__name__", "") if func is not None else "" + direct = public_name in public if direct and (params is None or isinstance(params, dict)): if params: kwargs = ", ".join( f"{key}={value!r}" for key, value in params.items()) - return f"ac.{func.__name__}({kwargs})" - return f"ac.{func.__name__}()" + return f"ac.{public_name}({kwargs})" + return f"ac.{public_name}()" return f"ac.execute_action({[list(action)]!r})" diff --git a/je_auto_control/utils/color_match/color_match.py b/je_auto_control/utils/color_match/color_match.py index c7dd7cd0..d37d3e99 100644 --- a/je_auto_control/utils/color_match/color_match.py +++ b/je_auto_control/utils/color_match/color_match.py @@ -50,6 +50,8 @@ def _score_map(template_hsv, haystack_hsv, channels: Sequence[str]): template_hsv[:, :, index], cv2.TM_SQDIFF_NORMED) result = np.nan_to_num(result, nan=1.0, posinf=1.0) accumulator = result if accumulator is None else accumulator + result + if accumulator is None: + raise ValueError("match_color needs at least one channel") return 1.0 - accumulator / len(channels) diff --git a/je_auto_control/utils/color_stats/color_stats.py b/je_auto_control/utils/color_stats/color_stats.py index a3839f19..6d44c80b 100644 --- a/je_auto_control/utils/color_stats/color_stats.py +++ b/je_auto_control/utils/color_stats/color_stats.py @@ -78,7 +78,8 @@ def region_color_stats(source: ImageSource, """ image = _load_rgb(source) if region is not None: - image = image.crop(tuple(int(v) for v in region)) + left, top, right, bottom = (int(v) for v in region) + image = image.crop((left, top, right, bottom)) image.thumbnail((128, 128)) pixels: List[RGB] = list(image.getdata()) count = len(pixels) diff --git a/je_auto_control/utils/config_sync/client.py b/je_auto_control/utils/config_sync/client.py index 59ec011a..bd231cde 100644 --- a/je_auto_control/utils/config_sync/client.py +++ b/je_auto_control/utils/config_sync/client.py @@ -96,7 +96,8 @@ def merge_buckets(local: ConfigBucket, local_entry = local_sec.get(entry_id) remote_entry = remote_sec.get(entry_id) if local_entry is None: - merged_section[entry_id] = remote_entry + if remote_entry is not None: + merged_section[entry_id] = remote_entry continue if remote_entry is None: merged_section[entry_id] = local_entry diff --git a/je_auto_control/utils/cv2_utils/video_recording.py b/je_auto_control/utils/cv2_utils/video_recording.py index 8df2d28f..8126ac21 100644 --- a/je_auto_control/utils/cv2_utils/video_recording.py +++ b/je_auto_control/utils/cv2_utils/video_recording.py @@ -48,7 +48,7 @@ def run(self): resolution = sct.monitors[0] output_file = self.video_name + ".mp4" - fourcc = cv2.VideoWriter_fourcc(*"mp4v") + fourcc = cv2.VideoWriter_fourcc(*"mp4v") # type: ignore[attr-defined] # reason: absent from the cv2 stub video_writer = cv2.VideoWriter( output_file, fourcc, diff --git a/je_auto_control/utils/dag/runner.py b/je_auto_control/utils/dag/runner.py index 08719d9d..a7094652 100644 --- a/je_auto_control/utils/dag/runner.py +++ b/je_auto_control/utils/dag/runner.py @@ -271,6 +271,9 @@ def _default_remote_runner(node: DagNode, def _resolve_remote_actions(node: DagNode) -> List[Any]: if node.actions is not None: return list(node.actions) + if node.action_file is None: + raise RuntimeError( + f"node {node.id!r} has neither actions nor an action_file") import json with open(node.action_file, "r", encoding="utf-8") as fp: loaded = json.load(fp) diff --git a/je_auto_control/utils/data_quality/data_quality.py b/je_auto_control/utils/data_quality/data_quality.py index 2bf56812..242924e9 100644 --- a/je_auto_control/utils/data_quality/data_quality.py +++ b/je_auto_control/utils/data_quality/data_quality.py @@ -9,7 +9,7 @@ """ import hashlib import re -from typing import Any, Dict, List, Optional +from typing import Any, Dict, List, Optional, Set, cast _TYPES = { "int": int, "float": (int, float), "number": (int, float), @@ -36,7 +36,7 @@ def _matches_type(value: Any, kind: str) -> bool: return True if kind in ("int", "number", "float") and isinstance(value, bool): return False - return isinstance(value, expected) + return isinstance(value, cast(type, expected)) def _number_range_error(value: Any, rule: Dict[str, Any]) -> Optional[str]: @@ -110,7 +110,8 @@ def validate_rows(rows: List[Dict[str, Any]], ``errors`` (``{row, field, error}``). """ rows = list(rows) - seen_unique = {field: set() for field, rule in schema.items() + seen_unique: Dict[str, Set[Any]] = { + field: set() for field, rule in schema.items() if rule.get("unique")} errors: List[Dict[str, Any]] = [] valid: List[Dict[str, Any]] = [] diff --git a/je_auto_control/utils/dbus_client/session_bus.py b/je_auto_control/utils/dbus_client/session_bus.py index d5777292..b6bf3f9f 100644 --- a/je_auto_control/utils/dbus_client/session_bus.py +++ b/je_auto_control/utils/dbus_client/session_bus.py @@ -452,7 +452,10 @@ def connect(self) -> None: ) path, abstract = _socket_target(self.address) try: - self._socket = socket.socket(socket.AF_UNIX, socket.SOCK_STREAM) + self._socket = socket.socket( + socket.AF_UNIX, # type: ignore[attr-defined] # reason: POSIX-only + socket.SOCK_STREAM, + ) self._socket.connect(("\0" + path) if abstract else path) except OSError as error: self.close() @@ -477,7 +480,7 @@ def sender_token(self) -> str: def _authenticate(self) -> None: """The SASL EXTERNAL handshake, which is a uid in hex over a socket.""" - uid = str(os.getuid()).encode("ascii") + uid = str(os.getuid()).encode("ascii") # type: ignore[attr-defined] # reason: POSIX-only self._send_raw(b"\x00AUTH EXTERNAL " + uid.hex().encode("ascii") + b"\r\n") reply = self._read_line() diff --git a/je_auto_control/utils/element_parse/element_parse.py b/je_auto_control/utils/element_parse/element_parse.py index fee457db..30470d99 100644 --- a/je_auto_control/utils/element_parse/element_parse.py +++ b/je_auto_control/utils/element_parse/element_parse.py @@ -74,7 +74,7 @@ def fuse_elements(ocr_boxes: Optional[Sequence[Box]] = None, item = dict(box) item.setdefault("source", source) tagged.append(item) - tagged.sort(key=lambda box: (rank.get(box.get("source"), len(rank)), + tagged.sort(key=lambda box: (rank.get(str(box.get("source", "")), len(rank)), -_area(box))) return _dedup(tagged, float(iou_threshold)) diff --git a/je_auto_control/utils/element_proposal/element_proposal.py b/je_auto_control/utils/element_proposal/element_proposal.py index 3b08b274..8912f9a9 100644 --- a/je_auto_control/utils/element_proposal/element_proposal.py +++ b/je_auto_control/utils/element_proposal/element_proposal.py @@ -34,7 +34,7 @@ def tag_kinds(elements: Sequence[Dict[str, Any]]) -> List[Dict[str, Any]]: for element in elements: box = [int(element["x"]), int(element["y"]), int(element["width"]), int(element["height"])] - kind = _KIND_BY_SOURCE.get(element.get("source"), "widget") + kind = _KIND_BY_SOURCE.get(str(element.get("source", "")), "widget") result.append({"box": box, "kind": kind, "index": element.get("index")}) return result diff --git a/je_auto_control/utils/expect_poll/expect_poll.py b/je_auto_control/utils/expect_poll/expect_poll.py index 5124e43c..7c97855c 100644 --- a/je_auto_control/utils/expect_poll/expect_poll.py +++ b/je_auto_control/utils/expect_poll/expect_poll.py @@ -12,7 +12,7 @@ """ import time from dataclasses import dataclass -from typing import Any, Callable +from typing import Any, Callable, Dict from je_auto_control.utils.exception.exceptions import AutoControlActionException @@ -63,7 +63,7 @@ def to_be_truthy() -> Matcher: def to_be_stable(times: int = 3) -> Matcher: """Match once the value has been equal across ``times`` consecutive polls.""" - state = {"last": object(), "count": 0} + state: Dict[str, Any] = {"last": object(), "count": 0} def matcher(value: Any) -> bool: if value == state["last"]: diff --git a/je_auto_control/utils/feature_match/feature_match.py b/je_auto_control/utils/feature_match/feature_match.py index ca44560b..40db686c 100644 --- a/je_auto_control/utils/feature_match/feature_match.py +++ b/je_auto_control/utils/feature_match/feature_match.py @@ -58,8 +58,9 @@ def _make_orb(template_gray, max_features: int): smaller = min(template_gray.shape[:2]) patch = max(7, min(31, smaller // 3)) edge = max(2, patch // 3) - return cv2.ORB_create(nfeatures=int(max_features), edgeThreshold=edge, - patchSize=patch) + return cv2.ORB_create(# type: ignore[attr-defined] # reason: absent from the cv2 stub + nfeatures=int(max_features), edgeThreshold=edge, + patchSize=patch) def _keypoint_matches(template_gray, scene_gray, max_features: int, ratio: float): diff --git a/je_auto_control/utils/flow_debugger/flow_debugger.py b/je_auto_control/utils/flow_debugger/flow_debugger.py index 2f7c4365..dbc6cc68 100644 --- a/je_auto_control/utils/flow_debugger/flow_debugger.py +++ b/je_auto_control/utils/flow_debugger/flow_debugger.py @@ -95,14 +95,20 @@ def continue_(self, max_steps: int = 100000) -> List[Dict[str, Any]]: while not self.finished and len(executed) < max_steps: if self._index in self._breakpoints and executed: break - executed.append(self.step()) + record = self.step() + if record is None: + break + executed.append(record) return executed def run_to_end(self) -> List[Dict[str, Any]]: """Run every remaining action, ignoring breakpoints.""" executed: List[Dict[str, Any]] = [] while not self.finished: - executed.append(self.step()) + record = self.step() + if record is None: + break + executed.append(record) return executed def reset(self) -> None: diff --git a/je_auto_control/utils/generate_report/generate_xml_report.py b/je_auto_control/utils/generate_report/generate_xml_report.py index aac64dfc..d113f6cc 100644 --- a/je_auto_control/utils/generate_report/generate_xml_report.py +++ b/je_auto_control/utils/generate_report/generate_xml_report.py @@ -18,11 +18,9 @@ def generate_xml() -> Tuple[Union[str, bytes], Union[str, bytes]]: autocontrol_logger.info("generate_xml") success_dict, failure_dict = generate_json() - success_dict = {"xml_data": success_dict} - failure_dict = {"xml_data": failure_dict} - success_xml = dict_to_elements_tree(success_dict) - failure_xml = dict_to_elements_tree(failure_dict) + success_xml = dict_to_elements_tree({"xml_data": success_dict}) + failure_xml = dict_to_elements_tree({"xml_data": failure_dict}) return success_xml, failure_xml diff --git a/je_auto_control/utils/humanize/motion.py b/je_auto_control/utils/humanize/motion.py index 09c277c5..48ae6117 100644 --- a/je_auto_control/utils/humanize/motion.py +++ b/je_auto_control/utils/humanize/motion.py @@ -11,6 +11,10 @@ from dataclasses import dataclass from typing import Callable, List, Optional, Tuple +from je_auto_control.utils.exception.exceptions import ( + AutoControlMouseException, +) + Point = Tuple[float, float] @@ -105,6 +109,9 @@ def move_mouse_humanized(x: int, y: int, *, duration_s: float = 0.4, ) motion = motion or HumanizedMotion() start = get_mouse_position() + if start is None: + raise AutoControlMouseException( + "the OS did not report a cursor position to move from") path = humanized_path(start, (x, y), motion) per_step = max(0.0, float(duration_s)) / max(1, len(path)) for point_x, point_y in path: diff --git a/je_auto_control/utils/json_patch/json_patch.py b/je_auto_control/utils/json_patch/json_patch.py index 2398a69e..70207022 100644 --- a/je_auto_control/utils/json_patch/json_patch.py +++ b/je_auto_control/utils/json_patch/json_patch.py @@ -227,7 +227,7 @@ def apply_patch(doc: Any, patch: List[Dict[str, Any]]) -> Any: """Apply an RFC 6902 patch to ``doc`` atomically; return the new document.""" result = copy.deepcopy(doc) for op in patch: - handler = _OPS.get(op.get("op")) + handler = _OPS.get(str(op.get("op", ""))) if handler is None: raise PatchError(f"unknown patch op {op.get('op')!r}") result = handler(result, op) diff --git a/je_auto_control/utils/locale_collation/locale_collation.py b/je_auto_control/utils/locale_collation/locale_collation.py index 340075d1..6779568a 100644 --- a/je_auto_control/utils/locale_collation/locale_collation.py +++ b/je_auto_control/utils/locale_collation/locale_collation.py @@ -14,7 +14,7 @@ CI and across platforms (unlike ``locale.strxfrm``). """ import unicodedata -from typing import Callable, Dict, List, Optional, Sequence, Tuple +from typing import Any, Callable, Dict, List, Optional, Sequence, Tuple _STRENGTHS = {"primary": 1, "secondary": 2, "tertiary": 3} @@ -113,7 +113,7 @@ def sort_strings(items: Sequence[str], *, strength: str = "tertiary", ``key`` extracts the string from each item (default: the item itself), so dicts or tuples can be sorted by one of their fields. """ - extract = key or (lambda item: item) + extract: Callable[[Any], str] = key or (lambda item: str(item)) def sort_key(item: object) -> CollationKey: return collation_key(str(extract(item)), strength=strength, diff --git a/je_auto_control/utils/monitor_layout/logical_frame.py b/je_auto_control/utils/monitor_layout/logical_frame.py index 04da7a5e..194717ca 100644 --- a/je_auto_control/utils/monitor_layout/logical_frame.py +++ b/je_auto_control/utils/monitor_layout/logical_frame.py @@ -95,7 +95,7 @@ def _backend_frame_origin() -> Tuple[int, int]: return backend_layout_origin() -def _load_image_grab(): +def _load_image_grab() -> Any: """Load the platform's ``ImageGrab``-shaped grabber lazily. Pillow off Wayland, the compositor's capture tool on it — see @@ -113,7 +113,7 @@ def _resample(): def grab_logical(region: Optional[Sequence[int]] = None, *, all_screens: bool = True, - grabber: Optional[Callable[..., Any]] = None, + grabber: Optional[Any] = None, metrics: Optional[MetricsReader] = None) -> Tuple[Any, int, int]: """Capture the screen in mouse-coordinate space. diff --git a/je_auto_control/utils/mouse_path/mouse_path.py b/je_auto_control/utils/mouse_path/mouse_path.py index 121ca93e..ea11afdf 100644 --- a/je_auto_control/utils/mouse_path/mouse_path.py +++ b/je_auto_control/utils/mouse_path/mouse_path.py @@ -33,7 +33,9 @@ def plan_path(waypoints: Sequence[Point], *, easing: str = "linear", first = waypoints[0] return [[int(first[0]), int(first[1])]] for index in range(len(waypoints) - 1): - segment = tween_points(waypoints[index], waypoints[index + 1], + here, there = waypoints[index], waypoints[index + 1] + segment = tween_points((int(here[0]), int(here[1])), + (int(there[0]), int(there[1])), per_segment_steps, easing) points.extend(segment[1:] if index else segment) return points diff --git a/je_auto_control/utils/ocr/backends/__init__.py b/je_auto_control/utils/ocr/backends/__init__.py index e29ee3a6..b836b7e6 100644 --- a/je_auto_control/utils/ocr/backends/__init__.py +++ b/je_auto_control/utils/ocr/backends/__init__.py @@ -51,6 +51,7 @@ def _build(name: str) -> OCRBackend: cached = _cached.get(name) if cached is not None: return cached + backend: OCRBackend if name == "tesseract": from je_auto_control.utils.ocr.backends.tesseract_backend import ( TesseractBackend, diff --git a/je_auto_control/utils/preprocess/preprocess.py b/je_auto_control/utils/preprocess/preprocess.py index 0f62e63e..9b0a1b81 100644 --- a/je_auto_control/utils/preprocess/preprocess.py +++ b/je_auto_control/utils/preprocess/preprocess.py @@ -11,7 +11,7 @@ OCR / match call or save. OpenCV + NumPy come in via the project's ``je_open_cv`` dependency and are imported lazily. Imports no ``PySide6``. """ -from typing import Any, Optional, Sequence +from typing import Any, Callable, Dict, Optional, Sequence ImageSource = Any _INTERP = ("nearest", "linear", "cubic", "lanczos") @@ -149,7 +149,7 @@ def _step_binarize(array, *, block_size: int, c: int, **_kwargs): return binarize(array, block_size=block_size, c=c) -_STEPS = { +_STEPS: Dict[str, Callable[..., Any]] = { "grayscale": _step_grayscale, "upscale": _step_upscale, "binarize": _step_binarize, diff --git a/je_auto_control/utils/profiler/resource_profiler.py b/je_auto_control/utils/profiler/resource_profiler.py index 813b4c35..bdc5460c 100644 --- a/je_auto_control/utils/profiler/resource_profiler.py +++ b/je_auto_control/utils/profiler/resource_profiler.py @@ -104,7 +104,9 @@ def start(self) -> None: return # FPS-only mode; no sampling thread needed # Warm up cpu_percent so the first real call returns a real number. try: - self._proc.cpu_percent(interval=None) + proc = self._proc + if proc is not None: + proc.cpu_percent(interval=None) except (AttributeError, OSError): pass self._thread = threading.Thread( diff --git a/je_auto_control/utils/qr/qr.py b/je_auto_control/utils/qr/qr.py index fb07c822..19d4c48c 100644 --- a/je_auto_control/utils/qr/qr.py +++ b/je_auto_control/utils/qr/qr.py @@ -30,7 +30,8 @@ def _load_np(source: ImageSource, region: Optional[Sequence[int]]): image = Image.open(str(source)) image = image.convert("RGB") if region is not None: - image = image.crop(tuple(int(v) for v in region)) + left, top, right, bottom = (int(v) for v in region) + image = image.crop((left, top, right, bottom)) return np.array(image) diff --git a/je_auto_control/utils/rate_limit/rate_limit.py b/je_auto_control/utils/rate_limit/rate_limit.py index 9c63f0b3..b8f0a6fa 100644 --- a/je_auto_control/utils/rate_limit/rate_limit.py +++ b/je_auto_control/utils/rate_limit/rate_limit.py @@ -17,7 +17,7 @@ import functools import threading import time -from typing import Callable, Optional +from typing import Callable, Dict, Optional from je_auto_control.utils.exception.exceptions import AutoControlException @@ -138,7 +138,7 @@ def throttle(interval_s: float, *, are dropped (the wrapper returns ``None``). """ def decorator(func: Callable) -> Callable: - state = {"last": None} + state: Dict[str, Optional[float]] = {"last": None} lock = threading.Lock() @functools.wraps(func) diff --git a/je_auto_control/utils/sbom/sbom.py b/je_auto_control/utils/sbom/sbom.py index 5f3ae282..91d4e709 100644 --- a/je_auto_control/utils/sbom/sbom.py +++ b/je_auto_control/utils/sbom/sbom.py @@ -31,7 +31,9 @@ def _component(dist: "metadata.Distribution") -> Dict[str, Any]: "type": "library", "name": name, "version": version, "purl": _purl(name, version), } - license_name = dist.metadata.get("License") + # `dist.metadata` is an `email.message.Message`: it answers `get`, + # but is not declared as a mapping. + license_name = dist.metadata.get("License") # type: ignore[attr-defined] # reason: Message.get if license_name and license_name != "UNKNOWN": component["licenses"] = [{"license": {"name": license_name}}] return component diff --git a/je_auto_control/utils/scheduler/scheduler.py b/je_auto_control/utils/scheduler/scheduler.py index 27ec183c..6ab7a584 100644 --- a/je_auto_control/utils/scheduler/scheduler.py +++ b/je_auto_control/utils/scheduler/scheduler.py @@ -223,7 +223,7 @@ def _fire(self, job: ScheduledJob, now_mono: float, now_wall: float) -> None: if live.max_runs is not None and live.runs >= live.max_runs: self._jobs.pop(job.job_id, None) return - if live.is_cron: + if live.is_cron and live.cron_expression is not None: next_dt = next_match(live.cron_expression, _dt.datetime.fromtimestamp(now_wall)) live.next_run_ts = next_dt.timestamp() diff --git a/je_auto_control/utils/search_index/search_index.py b/je_auto_control/utils/search_index/search_index.py index 4a97cb10..a8672d02 100644 --- a/je_auto_control/utils/search_index/search_index.py +++ b/je_auto_control/utils/search_index/search_index.py @@ -14,7 +14,9 @@ import re from collections import Counter from dataclasses import dataclass -from typing import Dict, Iterable, List, Optional, Sequence, Tuple, Union +from typing import ( + Dict, Iterable, List, Optional, Sequence, Set, Tuple, Union, +) _TOKEN_RE = re.compile(r"[a-z0-9]+") @@ -116,7 +118,7 @@ def search(self, query: str, *, top_k: int = 10, mode: str = "bm25") -> List[SearchHit]: """Return up to ``top_k`` documents ranked for ``query``.""" terms = self._terms(query) - candidates = set() + candidates: Set[str] = set() for term in terms: candidates.update(self._postings.get(term, {})) avgdl = self._avgdl() diff --git a/je_auto_control/utils/secrets/secret_store.py b/je_auto_control/utils/secrets/secret_store.py index 1dbe28b0..58288c2b 100644 --- a/je_auto_control/utils/secrets/secret_store.py +++ b/je_auto_control/utils/secrets/secret_store.py @@ -27,7 +27,7 @@ import os import threading from pathlib import Path -from typing import Dict, List, Optional +from typing import Any, Dict, List, Optional, Tuple _VERIFIER_PLAINTEXT = b"autocontrol-vault-v1" @@ -183,21 +183,21 @@ def set(self, name: str, value: str) -> None: if not isinstance(value, str): raise ValueError("secret value must be a string") with self._lock: - self._require_unlocked() - token = self._fernet.encrypt(value.encode("utf-8")).decode("ascii") - self._vault["items"][name] = token # type: ignore[index] - _atomic_write(self._path, self._vault) # type: ignore[arg-type] + fernet, vault = self._require_unlocked() + token = fernet.encrypt(value.encode("utf-8")).decode("ascii") + vault["items"][name] = token + _atomic_write(self._path, vault) def get(self, name: str) -> Optional[str]: """Return the plaintext for ``name`` or ``None`` if unset.""" with self._lock: - self._require_unlocked() - token = self._vault["items"].get(name) # type: ignore[index] + fernet, vault = self._require_unlocked() + token = vault["items"].get(name) if token is None: return None _, invalid_token = _fernet_types() try: - return self._fernet.decrypt(token.encode("ascii")).decode("utf-8") + return fernet.decrypt(token.encode("ascii")).decode("utf-8") except invalid_token as error: raise SecretStoreError( f"secret {name!r} failed integrity check" @@ -245,9 +245,12 @@ def destroy(self) -> None: except FileNotFoundError: pass - def _require_unlocked(self) -> None: - if self._fernet is None or self._vault is None: + def _require_unlocked(self) -> Tuple[Any, dict]: + """Return the live ``(fernet, vault)`` pair, or raise if locked.""" + fernet, vault = self._fernet, self._vault + if fernet is None or vault is None: raise SecretStoreLocked("secret vault is locked") + return fernet, vault default_secret_manager = SecretManager() diff --git a/je_auto_control/utils/socket_server/auto_control_socket_server.py b/je_auto_control/utils/socket_server/auto_control_socket_server.py index 715d1574..a85f89e2 100644 --- a/je_auto_control/utils/socket_server/auto_control_socket_server.py +++ b/je_auto_control/utils/socket_server/auto_control_socket_server.py @@ -35,7 +35,7 @@ def _read_command(request) -> str: return str(b"".join(chunks).strip(), encoding="utf-8") -def _close_server_async(server: socketserver.TCPServer) -> None: +def _close_server_async(server: socketserver.BaseServer) -> None: """Shut the server down and release its port, off the handler thread. 必須另開執行緒:ThreadingMixIn.server_close() 會 join 所有 handler diff --git a/je_auto_control/utils/text_normalize/text_normalize.py b/je_auto_control/utils/text_normalize/text_normalize.py index 628ca6b6..fd6666db 100644 --- a/je_auto_control/utils/text_normalize/text_normalize.py +++ b/je_auto_control/utils/text_normalize/text_normalize.py @@ -10,6 +10,7 @@ """ import re import unicodedata +from typing import Literal, Tuple, cast _QUOTE_MAP = { "‘": "'", "’": "'", "‚": "'", "‛": "'", @@ -20,6 +21,10 @@ _QUOTE_TABLE = str.maketrans(_QUOTE_MAP) +_NormalForm = Literal["NFC", "NFD", "NFKC", "NFKD"] +_NORMAL_FORMS: Tuple[str, ...] = ("NFC", "NFD", "NFKC", "NFKD") + + def fold_whitespace(text: str) -> str: """Collapse runs of whitespace to single spaces and strip the ends.""" return " ".join((text or "").split()) @@ -39,7 +44,11 @@ def normalize_quotes(text: str) -> str: def normalize_text(text: str, *, form: str = "NFKC", casefold: bool = True, collapse_ws: bool = True) -> str: """Canonicalise ``text``: Unicode ``form``, optional casefold + ws fold.""" - result = unicodedata.normalize(form, text or "") + if form not in _NORMAL_FORMS: + raise ValueError( + f"unknown normalisation form {form!r}; " + f"expected one of {', '.join(_NORMAL_FORMS)}") + result = unicodedata.normalize(cast(_NormalForm, form), text or "") if casefold: result = result.casefold() if collapse_ws: diff --git a/je_auto_control/utils/tls_acme/challenge.py b/je_auto_control/utils/tls_acme/challenge.py index 88896048..5c24650f 100644 --- a/je_auto_control/utils/tls_acme/challenge.py +++ b/je_auto_control/utils/tls_acme/challenge.py @@ -94,14 +94,15 @@ def start(self) -> int: return self._port def stop(self) -> None: - if not self.is_running: + server = self._server + if server is None: return try: - self._server.shutdown() + server.shutdown() except (OSError, RuntimeError): pass try: - self._server.server_close() + server.server_close() except OSError: pass if self._thread is not None: diff --git a/je_auto_control/utils/visual_match/visual_match.py b/je_auto_control/utils/visual_match/visual_match.py index b63c3b48..05f2bdb6 100644 --- a/je_auto_control/utils/visual_match/visual_match.py +++ b/je_auto_control/utils/visual_match/visual_match.py @@ -14,7 +14,7 @@ """ import functools from dataclasses import asdict, dataclass -from typing import Any, Dict, List, Optional, Sequence +from typing import Any, Dict, List, Optional, Protocol, Sequence, TypeVar from je_auto_control.utils.exception.exceptions import ( AutoControlFlatTemplateException, AutoControlScreenException, @@ -243,7 +243,34 @@ def match_template(template: ImageSource, *, haystack: Optional[ImageSource] = N return best -def _iou(a: Match, b: Match) -> float: +class _ScoredBox(Protocol): + """What suppression reads off a match — position, size, score. + + Read-only properties, not attributes: the match records are frozen + dataclasses, and a Protocol member declared as an attribute demands a + writable one. + """ + + @property + def x(self) -> int: ... + + @property + def y(self) -> int: ... + + @property + def width(self) -> int: ... + + @property + def height(self) -> int: ... + + @property + def score(self) -> float: ... + + +_BoxT = TypeVar("_BoxT", bound=_ScoredBox) + + +def _iou(a: _ScoredBox, b: _ScoredBox) -> float: left = max(a.x, b.x) top = max(a.y, b.y) right = min(a.x + a.width, b.x + b.width) @@ -255,8 +282,8 @@ def _iou(a: Match, b: Match) -> float: return inter / union -def _nms(matches: List[Match], iou_threshold: float) -> List[Match]: - kept: List[Match] = [] +def _nms(matches: List[_BoxT], iou_threshold: float) -> List[_BoxT]: + kept: List[_BoxT] = [] for candidate in sorted(matches, key=lambda m: m.score, reverse=True): if all(_iou(candidate, k) <= iou_threshold for k in kept): kept.append(candidate) diff --git a/je_auto_control/utils/visual_regression/compare.py b/je_auto_control/utils/visual_regression/compare.py index 12482002..e71a9839 100644 --- a/je_auto_control/utils/visual_regression/compare.py +++ b/je_auto_control/utils/visual_regression/compare.py @@ -4,7 +4,7 @@ import os from dataclasses import dataclass, field from pathlib import Path -from typing import TYPE_CHECKING, Optional, Sequence, Tuple +from typing import TYPE_CHECKING, Optional, Sequence, Tuple, cast if TYPE_CHECKING: # pragma: no cover - annotations only from PIL import Image @@ -129,11 +129,16 @@ def image_difference(actual: Image.Image, expected: Image.Image, overlay_draw = ImageDraw.Draw(overlay) differing = 0 diff_data = diff.load() + if diff_data is None: + raise ValueError("could not read the difference image") threshold = max(0, int(per_pixel_threshold)) width, height = diff.size for y in range(height): for x in range(width): - r, g, b = diff_data[x, y][:3] + # Both images were converted to RGB above, so a pixel is + # a three-int tuple rather than the union `load()` promises. + pixel = cast(Sequence[int], diff_data[x, y]) + r, g, b = pixel[0], pixel[1], pixel[2] if max(r, g, b) > threshold: differing += 1 overlay_draw.point((x, y), fill=(255, 0, 0)) diff --git a/je_auto_control/utils/watcher/watcher.py b/je_auto_control/utils/watcher/watcher.py index 8211458b..aa0b6b9d 100644 --- a/je_auto_control/utils/watcher/watcher.py +++ b/je_auto_control/utils/watcher/watcher.py @@ -17,9 +17,13 @@ def sample(self) -> Tuple[int, int]: """Return the current ``(x, y)``; raise ``RuntimeError`` on failure.""" from je_auto_control.wrapper.auto_control_mouse import get_mouse_position try: - x, y = get_mouse_position() + position = get_mouse_position() except (OSError, RuntimeError, ValueError, TypeError) as error: raise RuntimeError(f"MouseWatcher.sample failed: {error!r}") from error + if position is None: + # The Windows backend reports a failed GetCursorPos this way. + raise RuntimeError("MouseWatcher.sample: no cursor position") + x, y = position return int(x), int(y) diff --git a/je_auto_control/utils/xml/xml_file/xml_file.py b/je_auto_control/utils/xml/xml_file/xml_file.py index bd22b24f..da9fe6a4 100644 --- a/je_auto_control/utils/xml/xml_file/xml_file.py +++ b/je_auto_control/utils/xml/xml_file/xml_file.py @@ -84,6 +84,8 @@ def xml_parser_from_file(self, **kwargs) -> ElementTree.Element: except (OSError, ParseError) as error: raise XMLException(f"{cant_read_xml_error_message}: {repr(error)}") from error self.xml_root = self.tree.getroot() + if self.xml_root is None: + raise XMLException(f"{cant_read_xml_error_message}: empty document") self.xml_from_type = "file" return self.xml_root diff --git a/je_auto_control/wrapper/auto_control_screen.py b/je_auto_control/wrapper/auto_control_screen.py index 82b7517a..93ed0491 100644 --- a/je_auto_control/wrapper/auto_control_screen.py +++ b/je_auto_control/wrapper/auto_control_screen.py @@ -1,5 +1,5 @@ import sys -from typing import List, Optional, Tuple +from typing import Any, Optional, Tuple from je_auto_control.utils.cv2_utils.optional import require_cv2 from je_auto_control.utils.cv2_utils.screenshot import pil_screenshot @@ -31,7 +31,7 @@ def screen_size() -> Tuple[int, int]: def screenshot(file_path: Optional[str] = None, - screen_region: Optional[list] = None) -> List[int]: + screen_region: Optional[list] = None) -> Any: """ use to capture current screen 擷取當前螢幕畫面 diff --git a/test/verify/typing_contract_exempt.txt b/test/verify/typing_contract_exempt.txt index 3821e187..7aa1dedb 100644 --- a/test/verify/typing_contract_exempt.txt +++ b/test/verify/typing_contract_exempt.txt @@ -14,47 +14,6 @@ # "Does not type-check" means on at least one of the three targets mypy can be # pointed at (win32, linux, darwin) — not just the one CI happens to run on. # -# Measured entries: 42 +# Measured entries: 1 -je_auto_control.__main__ -je_auto_control.cli je_auto_control.gui.remote_desktop.webrtc_panel -je_auto_control.utils.act_modes.act_modes -je_auto_control.utils.annotate.annotate -je_auto_control.utils.change_localize.change_localize -je_auto_control.utils.codegen.codegen -je_auto_control.utils.color_match.color_match -je_auto_control.utils.color_stats.color_stats -je_auto_control.utils.config_sync.client -je_auto_control.utils.cv2_utils.screen_record -je_auto_control.utils.cv2_utils.video_recording -je_auto_control.utils.dag.runner -je_auto_control.utils.data_quality.data_quality -je_auto_control.utils.dbus_client.session_bus -je_auto_control.utils.element_parse.element_parse -je_auto_control.utils.element_proposal.element_proposal -je_auto_control.utils.expect_poll.expect_poll -je_auto_control.utils.feature_match.feature_match -je_auto_control.utils.flow_debugger.flow_debugger -je_auto_control.utils.generate_report.generate_xml_report -je_auto_control.utils.humanize.motion -je_auto_control.utils.json_patch.json_patch -je_auto_control.utils.locale_collation.locale_collation -je_auto_control.utils.monitor_layout.logical_frame -je_auto_control.utils.mouse_path.mouse_path -je_auto_control.utils.ocr.backends -je_auto_control.utils.preprocess.preprocess -je_auto_control.utils.profiler.resource_profiler -je_auto_control.utils.qr.qr -je_auto_control.utils.rate_limit.rate_limit -je_auto_control.utils.rotated_match.rotated_match -je_auto_control.utils.sbom.sbom -je_auto_control.utils.scheduler.scheduler -je_auto_control.utils.search_index.search_index -je_auto_control.utils.secrets.secret_store -je_auto_control.utils.socket_server.auto_control_socket_server -je_auto_control.utils.text_normalize.text_normalize -je_auto_control.utils.tls_acme.challenge -je_auto_control.utils.visual_regression.compare -je_auto_control.utils.watcher.watcher -je_auto_control.utils.xml.xml_file.xml_file From 080a434d9268cfde19aa19737b5be1fe6aac08b8 Mon Sep 17 00:00:00 2001 From: JeffreyChen Date: Sat, 22 Aug 2026 22:13:28 +0800 Subject: [PATCH 19/19] Empty the typing contract's exemption list MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit `je_auto_control` type-checks clean on win32, linux and darwin with nothing exempted. The list that started this branch at 155 modules now holds a header and no entries, and the gate fails if it grows again. The last module was webrtc_panel.py, blocked by its size rather than its types. Seven of its errors came from `_build_advanced_group(panel: TranslatableMixin, …)`, a free function that writes five widget attributes back onto the panel it is handed — none of which TranslatableMixin has. Saying that needs a Protocol, and the file was sitting exactly on the 2,555-line cap it may only shrink from. So the builder moved out, which is the split Progress.md already said that file owed: `advanced_group.py` holds the shared STUN/TURN group, the `AdvancedGroupHost` protocol naming what it reads and what it sets, and the hardware-codec row as its own function. The panel is 2,545 lines — under its cap for the first time — and both panels were rebuilt offscreen to confirm the STUN default, the TURN fields and the host-only codec picker still arrive where they did. Ten more errors there were the pattern the whole sweep kept meeting: a handle that is None until the session starts. `_produce_offer`, `_trust_session_viewer`, `_answer_and_push`, `_produce_answer` and the folder sync all reached through `_multi_host` / `_viewer` without asking. They go through `_require_multi_host()` / `_require_viewer()` now, raising a translated "start hosting first" / "connect to a host first" rather than an AttributeError on None — two new keys in all four language catalogues. Progress.md's mypy entry collapses to what is still worth knowing: the five recurring shapes, that a catch tuple has to be an annotated module constant, that `# type: ignore` only counts as a line's first comment, and that the cv2 stub omits names the library has. --- Progress.md | 160 ++++-------------- WHATS_NEW.md | 32 ++++ architecture_explore.md | 13 +- .../gui/language_wrapper/english.py | 2 + .../gui/language_wrapper/japanese.py | 2 + .../language_wrapper/simplified_chinese.py | 2 + .../language_wrapper/traditional_chinese.py | 2 + .../gui/remote_desktop/advanced_group.py | 92 ++++++++++ .../gui/remote_desktop/webrtc_panel.py | 96 +++++------ test/verify/typing_contract_exempt.txt | 3 +- 10 files changed, 216 insertions(+), 188 deletions(-) create mode 100644 je_auto_control/gui/remote_desktop/advanced_group.py diff --git a/Progress.md b/Progress.md index f295fa4b..c0a73398 100644 --- a/Progress.md +++ b/Progress.md @@ -18,12 +18,13 @@ `CLAUDE.md` §Size and complexity limits 規定:超標檔案只能列在這裡,列不進來的就是缺陷。 清單上的檔案**可以改、可以變短,但不得再變長**——要再長就得先拆。 -行數為 2026-08-19 實測(`len(text.splitlines())`)。 +行數為 2026-08-19 實測(`len(text.splitlines())`);`webrtc_panel.py` 於 +2026-08-22 拆出 `advanced_group.py` 後降到 2,545,上限跟著往下走。 | 檔案 | 行數 | 為何還沒拆 | | --- | ---: | --- | | `utils/mcp_server/tools/_handlers.py` | 4,789 | 676 個 MCP 工具的處理函式本體。與 `_factories.py`(表)不同,這裡是邏輯,應該依主題拆成 `_handlers/` 套件(input/screen/window/file/agent…)。拆點清楚,純粹是量大。 | -| `gui/remote_desktop/webrtc_panel.py` | 2,555 | 單一 Qt 面板,但已含連線、監視器選擇、頻寬自適應、麥克風、錄影五組互動狀態。應拆成 panel + 各控制器。 | +| `gui/remote_desktop/webrtc_panel.py` | 2,545 | 單一 Qt 面板,但已含連線、監視器選擇、頻寬自適應、麥克風、錄影五組互動狀態。應拆成 panel + 各控制器。 | | `utils/accessibility/backends/windows_backend.py` | 915 | 已拆出 `windows_query.py`(170)與 `windows_state.py`(98)。剩下的是同一套 UIA COM 生命週期管理,再拆會把 `CoInitialize`/介面釋放的配對邏輯切散。 | **本質豁免(依 `CLAUDE.md` 的「flat data tables」條款,不算既有豁免)**: @@ -39,7 +40,7 @@ 2026-08-18 重新實測時,表上原有的七列**全部**變長,而 `CLAUDE.md` 明寫 「列上的檔案不得再變長,要再長就得先拆」,所以這裡曾標成 `[DECIDE]`。 **維護者已於 2026-08-19 拍板:接受實測數字當新基準**——不為了回到舊數字而去拆 -`_handlers.py`(4,789)與 `webrtc_panel.py`(2,555)。上表的行數即是各自的新上限, +`_handlers.py`(4,789)與 `webrtc_panel.py`。上表的行數即是各自的新上限, 規則不變:只准變短,再變長就得先拆。 同一批裡有六個檔案在 2026-08-19 已經拆回線內、從表上移除,做法寫在 @@ -219,13 +220,14 @@ capability enum 值與 variadic `ei_seat_bind_capabilities`、event-type enum --- -## 兩個門檻:機制已經有了,剩下的是把數字往前推 +## 兩個門檻:mypy 那半已經到終點,覆蓋率那半還在爬 -`TODO` — 兩者都已經從「講好要爬」變成「有棘輪在擋」,但都還沒到目的地 +`TODO` — 只剩覆蓋率;型別契約 2026-08-22 收工 原本這一條記的是兩個只存在於 `pyproject.toml` 註解裡、沒有任何機制的承諾。 -2026-08-21 把**機制**補上了(做法見 [WHATS_NEW.md](WHATS_NEW.md)),所以這裡改記 -還差多少、以及下一步怎麼走。 +2026-08-21 把**機制**補上了(做法見 [WHATS_NEW.md](WHATS_NEW.md))。 +型別契約已於 2026-08-22 走完(豁免清單清空,見下),所以這條剩下的實質內容 +只有覆蓋率;mypy 那一節留著是因為它記的那幾個坑之後還會踩到。 ### 覆蓋率:地板 50,目標 70 @@ -240,129 +242,33 @@ capability enum 值與 variadic `ei_seat_bind_capabilities`、event-type enum (`--cov-report=term-missing` 已經開著,CI 每一格都存了 `coverage.xml` artifact), 而不是齊頭式地補測試。 -### mypy:整包把關,56 個模組還沒過 +### mypy:整包把關,**豁免清單已經清空** + +`TODO` → **完成(2026-08-22)** 範圍不再是兩條路徑,而是**整包減去一張只准變少的清單** (`test/verify/typing_contract_exempt.txt`)。差別在於預設值:路徑清單只有人想到才會長, -新模組預設在圈外;現在新模組**預設就在契約裡**,1,017 個檔案有 961 個已經過關。 - -`wrapper` 那一群(6 個模組)在 2026-08-21 清掉了,做法見 [WHATS_NEW.md](WHATS_NEW.md): -平台縫的八個匯出名稱先宣告型別、再讓分支去綁定,其中三個用 -`wrapper/backend_contract.py` 的 Protocol,四個 `_platform_*` 組裝模組各自標注自己綁了什麼。 -順帶帶綠了四個原本卡在縫的偶然型別上、自己其實沒問題的模組 -(`utils/cv2_utils/screen_grabber`、`utils/executor/mouse_aliases`、 -`utils/pytest_plugin/keywords`、`utils/vision/vlm_api`)。 - -`linux_wayland` 那一群(4 個模組)在 2026-08-22 清掉了:`libei` 的 33 處 -`Optional[BoundSymbols]` 改走一個會拋 `LibeiUnavailable` 的存取器(`_api`), -`_detect` 的兩支環境查詢收 `Mapping[str, str]` 而不是 `dict`(預設值就是 `os.environ`), -`capture._write_to_temp_png` 承認它的 writer 會回傳東西而它不看, -`screen.get_pixel` 把 Pillow 那個「每種 mode 的答案聯集」收斂成它自己承諾的三元組。 -做法見 [WHATS_NEW.md](WHATS_NEW.md)。 - -`linux_with_x11`(3)與 `osx`(1)同日清掉:X11 listener 的 `record_queue` -從「型別是 None」變成 `Optional[Queue]`(`stop_record()` 因此不再回一個 -簽章不承認的 `None`),`uinput/_device` 的 POSIX 專屬 `O_NONBLOCK` 收進一個 -mypy 剪得掉的 `sys.platform` 分支,`osx_keyboard` 的字串 keycode 一律當特殊鍵名。 - -`windows/` 這一群的**真實錯誤全部清掉了**:`window_message` 匯入一個 -`windows_window_manage` 根本沒有匯出的名字(`FindWindowW`),所以那個模組在 -Windows 上一直是 import 就炸;`win32_ctype_input` 則有一批標錯的註記 -(`_fields_: tuple` 把 ClassVar 重新宣告成 instance variable、`ctypes.POINTER` -與 `user32.SendInput` 被當成型別用)以及一行改寫標準函式庫的 -`wintypes.ULONG_PTR = wintypes.WPARAM`——全樹沒有任何地方讀它。 -**以 `--platform win32` 檢查時,`je_auto_control/windows/` 底下已經零錯誤。** -剩下的豁免是下面那條 `DECIDE`。 - -`utils/remote_desktop` 那一群(13 個模組、169 個錯誤,曾是最大的一群)在 -2026-08-22 清掉了,錯誤只有兩種形狀:九個模組是 `self._x = None` 沒有標注 -(mypy 就把那個屬性的**型別**判成 `None`),其中好幾個把想要的型別寫在行尾註解裡 -——事實一直都在,只是寫在沒有檢查器讀得到的地方;另外四個是 mixin 讀取 -自己不擁有的屬性,而每個 mixin 的 docstring 早就列出它跟宿主借了什麼, -現在那份清單變成類別本體裡的 `if TYPE_CHECKING:` 宣告(執行期會被剝掉, -所以不可能蓋掉宿主真正綁的東西)。順帶修掉三個行為問題: -`WebRTCLoopBridge._run` 從共用狀態讀 loop、`_get_cursor_position` 在函式內 -`import sys as _sys` 導致 mypy 剪不掉平台分支、`totp` 接的是 -`base64.binascii.Error`(那個名字只是 `base64` 自己 import 的副作用)。 -做法見 [WHATS_NEW.md](WHATS_NEW.md)。 - -同日再清掉十五個:`utils/accessibility` 那一群(5)、`utils/observability`(3)、 -`utils/triggers`(3)、`utils/chatops/router`、`utils/rest_api/rest_server`、 -`utils/mcp_server/http_transport`、`utils/element_repository`。四十三個 -accessibility 錯誤裡有三十四個是同一個沒標的回傳型別(`_unsupported` 一定會拋, -標成 `NoReturn` 就全清);另一個反覆出現的是**攔截用的 tuple 沒有被標成 tuple** -(`Tuple[type, ...]` 不算「a tuple of exception classes」, -`except (…, *SQLITE_ERRORS)` 的星號解包 mypy 也跟不進 `except`)。 -順帶修掉四個行為問題:`Gauge`/`Histogram` 用 -`_labels_key = Counter._labels_key` 借方法、`parse_content_length` 宣告的 -`Mapping[str, str]` 三個呼叫端一個都沒給過(給的是 `email.message.Message`)、 -`ElementRepository` 把使用者可編輯的 locator 直接 `**kwargs` 丟進無障礙 API、 -`_AtspiConnection._call` 在 `with` 之外沒有 bus 可以呼叫。 -做法見 [WHATS_NEW.md](WHATS_NEW.md)。 - -下面那條 `DECIDE` 也在同日拍板收掉,又清掉 16 個模組(做法見 -[WHATS_NEW.md](WHATS_NEW.md))。 - -ctypes 表面一收掉,卡在它後面的兩群也跟著清了:`utils/usb/passthrough`(3) -與 `utils/clipboard`(3)。兩邊都藏著真的缺陷——`winusb_backend._load_dlls` -一次公布一個 handle,第二個載入失敗時第一個已經設好,於是 `is not None` 的守衛 -把之後每一次重試都短路掉;`clipboard_api()` 宣告回傳 `Tuple[object, object]`, -而 `object` 沒有任何屬性,所以透過它的十二個 Win32 呼叫全是型別錯誤。 -做法見 [WHATS_NEW.md](WHATS_NEW.md)。 - -`gui` 那一群(13 個模組)同日清掉,只剩 `webrtc_panel.py`(見下)。 -大宗又是 mixin 借用宿主的成員(六個分頁 mixin 借 `_tr`/`_translate`/`timer`), -做法與 webrtc 那三個 mixin 相同。順帶修掉三個真的會壞的地方: -`assertions_tab` 的 pixel 斷言把 `*_parse_ints(...)[:2]` 星號展開,使用者只打一個 -座標時 RGB 清單會被綁到 `y`,錯誤訊息變成「關鍵字參數重複」而不是「你少打了一個 -座標」;`_auto_click_tab._get_mouse_pos` 直接解包 `get_mouse_position()`,而 -Windows 後端在鎖定桌面上真的會回 `None`;`multi_language_wrapper` 的監聽器清單 -標成 `List[callable]`(`callable` 是內建函式,不是型別)。 -做法見 [WHATS_NEW.md](WHATS_NEW.md)。 - -`utils/mcp_server` 那一群(4 個模組)同日清掉,順帶抓到一個**從來沒成功過的指令**: -`AC_normalize_url`/`ac_normalize_url` 都把 `drop_fragment=` 傳給 -`url_canon.normalize_url`,而那支函式的參數叫 `strip_fragment`——每一次呼叫都是 -`TypeError`。對外的名字不動(動作 schema 與工具註冊表都用它),改的是兩個呼叫點。 -做法見 [WHATS_NEW.md](WHATS_NEW.md)。 - -長尾那一批(13 個模組、每個 2~6 個錯誤)同日清掉,反覆出現的只有三種形狀: -`callable` 被當成型別用(`callable` 是內建函式)、`x: SomeType = None` 的隱含 -Optional、以及「掉了長度的 tuple」(`tuple(r)`/`cv2.boundingRect()` 都是 -`tuple[int, ...]`,而收它的欄位承諾四個 int)。另外抓到兩個只靠呼叫圖成立的不變量 -(`_StabilityTracker` 的 `now - self._since`、`act_when_ready` 把可能是 `None` 的 -`report.point` 傳給要求座標的 callback)。做法見 [WHATS_NEW.md](WHATS_NEW.md)。 - -**順帶一個 mypy 設定上的事實**:`pyproject.toml` 把 `cv2` 列在 -「ship no stubs 的基礎相依」底下,但 opencv-python **有**附 `.pyi`,所以閘門會去讀它, -而它的內容會隨 OpenCV 版本變動(實測 4.13.0:`cv2.MSER_create` 執行期存在、stub 裡沒有)。 -`>=4.8,<6` 這個範圍內版本一換,閘門的判定就可能跟著動——與 numpy 那條註解同一類的坑。 - -剩下 56 個模組要清,**不再有大群集**:最大的檔案是 -`gui/remote_desktop/webrtc_panel.py`(27,見下)與 -`utils/executor/action_executor.py`(8),其餘每個檔都只剩 1~2 個錯誤。 - -#### `webrtc_panel.py` 的型別豁免現在卡在它欠的那次拆檔 - -`TODO` — 27 個錯誤,但清掉它需要先讓這個檔可以變長 - -二十七個錯誤裡有七個來自 `_build_advanced_group(panel: TranslatableMixin, ...)`: -這個自由函式**會把五個 widget 屬性寫回 panel 上**(`_stun_edit`、`_turn_edit`、 -`_turn_user_edit`、`_turn_cred_edit`、`_hw_codec_combo`),而 `TranslatableMixin` -一個都沒有。正確的型別是一個 Protocol,把它需要與寫入的成員寫出來——但 -`webrtc_panel.py` **正好卡在 2,555 行的上限**(見本檔開頭那張表:只准變短), -而加 Protocol 一定變長。 - -真正的解法就是那張表早就寫著的:把這個檔拆成 panel + 各控制器。 -`_build_advanced_group` 本身就是第一個該搬出去的——它是共用的 widget group builder, -不屬於 panel 模組,搬走同時解決長度與型別兩件事。**在那之前這個模組留在豁免清單上, -理由記在這裡,不是被忘記。** -**下一步**是一次清一個群集,清完把行從清單刪掉—— -`python test/verify/typing_contract_verify.py --fix` 會替你改,CI 會在你忘了刪的時候紅掉。 -已經清掉的那些留下四個可以直接套用的樣板:mixin 用 `if TYPE_CHECKING:` 宣告 -借來的成員、`Optional` 屬性用 `TYPE_CHECKING` 匯入真正的型別去標注、 -一定會拋的 helper 標 `NoReturn`、攔截用的 tuple 標 -`Tuple[Type[BaseException], ...]` 並收成一個模組常數(不要在 `except` 裡星號解包)。 +新模組預設在圈外;現在新模組**預設就在契約裡**。 + +**2026-08-22 那張清單降到零**:`je_auto_control/` 的 1,018 個檔案在 +win32/linux/darwin 三個目標上全部乾淨。清掉 136 個模組的過程與每一群的做法寫在 +[WHATS_NEW.md](WHATS_NEW.md);這裡只留下之後還用得到的四件事: + +* **反覆出現的五種形狀**:mixin 讀取宿主的成員(用類別本體裡的 + `if TYPE_CHECKING:` 宣告,執行期會被剝掉)、`self._x = None` 沒有標注 + (mypy 會把屬性的型別判成 `None`)、`callable` 被當成型別用、 + `x: SomeType = None` 的隱含 Optional、以及掉了長度的 tuple。 +* **攔截用的 tuple 必須標成 `Tuple[Type[BaseException], ...]`**,而且要收成一個 + 模組常數——`except (A, B, *TUPLE)` 的星號解包 mypy 跟不進 `except`。 +* **`# type: ignore` 只有當它是那一行的第一個註解時才生效**(已實測),所以有 + `# nosec` 的行要把它放前面。 +* **`cv2` 的 stub 會隨版本變**:`pyproject.toml` 把它列在「ship no stubs 的基礎相依」 + 底下,但 opencv-python 有附 `.pyi`,閘門會去讀。實測 4.13.0:`MSER_create`、 + `ORB_create`、`VideoWriter_fourcc` 執行期都在、stub 裡都沒有。`>=4.8,<6` 範圍內 + 版本一換,判定就可能跟著動——與 numpy 那條註解同一類的坑。 + +清單現在只有標頭、沒有任何條目。**它變長就是退步**,`typing_contract_verify.py` +會在有人讓它變長時紅掉。 #### 平台縫還缺的一半:`keyboard` 與 `mouse` 還沒有合約 diff --git a/WHATS_NEW.md b/WHATS_NEW.md index 03dae6b3..2e8cd602 100644 --- a/WHATS_NEW.md +++ b/WHATS_NEW.md @@ -208,6 +208,38 @@ comparison as a `DECIDE`, because the cleanest of them changes what the gate means rather than what the code says. **136 modules left, 881 of 1,017 files inside the contract.** +### The Typing Contract's Exemption List Is Empty + +`je_auto_control` type-checks clean on all three targets — win32, linux and +darwin — with nothing exempted. The list that started this branch at 155 modules +now holds a header and no entries, and `typing_contract_verify.py` fails if it +ever grows again. + +The last module was `gui/remote_desktop/webrtc_panel.py`, and it was blocked by +its own size rather than by its types. Seven of its errors came from +`_build_advanced_group(panel: TranslatableMixin, …)`, a free function that +*writes* five widget attributes back onto the panel it is handed — none of which +`TranslatableMixin` has. Writing that contract down needs a Protocol, and the +file was sitting exactly on the 2,555-line cap it may only shrink from. + +So the builder moved out, which is what `Progress.md` had said that file owed +anyway: `gui/remote_desktop/advanced_group.py` now holds the shared +STUN/TURN group, the `AdvancedGroupHost` protocol naming what it reads and what +it sets, and the hardware-codec row as its own function. The panel is 2,545 +lines — under its cap for the first time — and both panels were rebuilt +offscreen afterwards to confirm the STUN default, the TURN fields and the +host-only codec picker all still arrive where they did. + +Ten more errors in that file were the pattern the whole sweep kept meeting: a +handle that is `None` until the session starts. `_produce_offer`, +`_trust_session_viewer`, `_answer_and_push`, `_produce_answer` and the folder +sync all reached through `self._multi_host` / `self._viewer` without asking. They +go through `_require_multi_host()` / `_require_viewer()` now, which raise a +translated "start hosting first" / "connect to a host first" instead of an +`AttributeError` on `None` — two new keys in all four language catalogues. + +**0 modules left. 1,018 of 1,018 files inside the contract.** + ### Thirteen Small Modules, and a Stub That Disagrees With the Library Past the big clusters the list is a long tail: thirteen modules of two to six diff --git a/architecture_explore.md b/architecture_explore.md index bb4e27be..3f6f99b4 100644 --- a/architecture_explore.md +++ b/architecture_explore.md @@ -19,8 +19,8 @@ iOS(WebDriverAgent)。核心能力是滑鼠/鍵盤控制、影像辨識、 | 指標 | 數值 | | --- | ---: | -| Python 模組總數(含周邊子專案) | 1,031 | -| 程式碼總行數 | 140,989 | +| Python 模組總數(含周邊子專案) | 1,032 | +| 程式碼總行數 | 141,079 | | `je_auto_control/utils/` 子套件數 | 310 | | `AC_*` 動作指令數(`known_commands()` 實測) | 773 | | 套件門面 `__all__` 公開名稱數 | 1,238 | @@ -930,12 +930,13 @@ GUI 是**選用 extra**(`pip install je_auto_control[gui]`,PySide6 + qt-mate | diagnostics | `diagnostics_tab.py` | 91 | 執行子系統檢查並顯示結果。 | | report | `_report_tab.py` | 81 | 產生 HTML/JSON/XML 報表。 | -#### 遠端桌面 GUI(`gui/remote_desktop/`,17 檔/6,254 行) +#### 遠端桌面 GUI(`gui/remote_desktop/`,18 檔/6,336 行) | 模組 | 行數 | 職責 | | --- | ---: | --- | -| `webrtc_panel.py` | 2,555 | WebRTC 子分頁主體。 | +| `webrtc_panel.py` | 2,545 | WebRTC 子分頁主體。 | | `webrtc_dialogs.py` | 493 | WebRTC GUI 用的自訂對話框與清單元件(待審檢視者、信任清單、通訊錄、遠端檔案表、稽核記錄、LAN 瀏覽)。 | +| `advanced_group.py` | 92 | 兩個 WebRTC 面板共用的 Advanced STUN/TURN(含選用硬體編碼器)群組,含它寫回面板的 Protocol。 | | `connection_screen.py` | 672 | Quick Connect —— AnyDesk 風格單畫面入口。 | | `viewer_panel.py` | 542 | 「控制另一台機器」子分頁。 | | `webrtc_known_hosts.py` | 340 | TOFU 釘選庫瀏覽器:`KnownHostsDialog` 與帶外釘選用的小表單。由 `webrtc_dialogs` 再匯出。 | @@ -1024,7 +1025,7 @@ socket 預設綁 `127.0.0.1`;資源一律用 `with`。 | 層/子系統 | 檔案數 | 行數 | | --- | ---: | ---: | -| `gui/` | 89 | 26,609 | +| `gui/` | 90 | 26,699 | | `utils/mcp_server/` | 21 | 17,354 | | `utils/remote_desktop/` | 56 | 11,990 | | `utils/executor/` | 6 | 9,081 | @@ -1045,5 +1046,5 @@ socket 預設綁 `127.0.0.1`;資源一律用 `with`。 | `autocontrol-lsp/` | 8 | 744 | | `utils/hotkey/` | 7 | 727 | | 其餘模組(約 286 個 `utils/` 子套件 + `android/`/`ios/`/周邊小工具) | 673 | 47,667 | -| **總計** | **1,025** | **140,924** | +| **總計** | **1,026** | **141,014** | diff --git a/je_auto_control/gui/language_wrapper/english.py b/je_auto_control/gui/language_wrapper/english.py index e6b4cb67..fab2fda0 100644 --- a/je_auto_control/gui/language_wrapper/english.py +++ b/je_auto_control/gui/language_wrapper/english.py @@ -574,6 +574,8 @@ "rd_webrtc_sync_start": "Start sync", "rd_webrtc_sync_stop": "Stop sync", "rd_webrtc_sync_dir_required": "Pick a local folder first", + "rd_webrtc_not_started": "Start hosting first", + "rd_webrtc_not_connected": "Connect to a host first", "rd_webrtc_browse": _BROWSE, # Auto Click Tab diff --git a/je_auto_control/gui/language_wrapper/japanese.py b/je_auto_control/gui/language_wrapper/japanese.py index 970d3bb5..846fa63d 100644 --- a/je_auto_control/gui/language_wrapper/japanese.py +++ b/je_auto_control/gui/language_wrapper/japanese.py @@ -463,6 +463,8 @@ "rd_webrtc_sync_start": "同期開始", "rd_webrtc_sync_stop": "同期停止", "rd_webrtc_sync_dir_required": "ローカルフォルダを選択してください", + "rd_webrtc_not_started": "先にホストを開始してください", + "rd_webrtc_not_connected": "先にホストへ接続してください", "rd_webrtc_browse": _BROWSE, # Auto Click Tab diff --git a/je_auto_control/gui/language_wrapper/simplified_chinese.py b/je_auto_control/gui/language_wrapper/simplified_chinese.py index a0733ef8..8788c7af 100644 --- a/je_auto_control/gui/language_wrapper/simplified_chinese.py +++ b/je_auto_control/gui/language_wrapper/simplified_chinese.py @@ -452,6 +452,8 @@ "rd_webrtc_sync_start": "开始同步", "rd_webrtc_sync_stop": "停止同步", "rd_webrtc_sync_dir_required": "请先选一个本机目录", + "rd_webrtc_not_started": "请先开始主机端", + "rd_webrtc_not_connected": "请先连线到主机", "rd_webrtc_browse": _BROWSE, # Auto Click Tab diff --git a/je_auto_control/gui/language_wrapper/traditional_chinese.py b/je_auto_control/gui/language_wrapper/traditional_chinese.py index c106b490..e7bd56b9 100644 --- a/je_auto_control/gui/language_wrapper/traditional_chinese.py +++ b/je_auto_control/gui/language_wrapper/traditional_chinese.py @@ -453,6 +453,8 @@ "rd_webrtc_sync_start": "開始同步", "rd_webrtc_sync_stop": "停止同步", "rd_webrtc_sync_dir_required": "請先選一個本機資料夾", + "rd_webrtc_not_started": "請先開始主機端", + "rd_webrtc_not_connected": "請先連線到主機", "rd_webrtc_browse": _BROWSE, # Auto Click Tab diff --git a/je_auto_control/gui/remote_desktop/advanced_group.py b/je_auto_control/gui/remote_desktop/advanced_group.py new file mode 100644 index 00000000..ac248ef5 --- /dev/null +++ b/je_auto_control/gui/remote_desktop/advanced_group.py @@ -0,0 +1,92 @@ +"""The 'Advanced' STUN/TURN group both WebRTC panels build. + +Split out of ``webrtc_panel`` for two reasons at once. It is a shared widget +builder rather than part of either panel, and it *writes* the five edits it +creates back onto the panel it is given — a contract that has to be written +down somewhere, and could not be written down inside a file already sitting on +its line-count cap. + +The panel argument is therefore a :class:`AdvancedGroupHost`: what the builder +reads (``_tr``, ``_on_hw_codec_changed``) and what it sets (the five widgets). +Imports ``PySide6`` — it is GUI-only by construction. +""" +from __future__ import annotations + +from typing import Any, Protocol + +from PySide6.QtWidgets import ( + QComboBox, QGridLayout, QGroupBox, QLabel, QLineEdit, QWidget, +) + +from je_auto_control.gui.remote_desktop._helpers import _t +from je_auto_control.utils.remote_desktop import ( + active_hardware_codec, available_hardware_codecs, +) + +DEFAULT_STUN = "stun:stun.l.google.com:19302" + + +class AdvancedGroupHost(Protocol): + """A WebRTC panel, seen from the advanced group it hosts. + + ``_tr`` and ``_on_hw_codec_changed`` are read; the five widget + attributes are *assigned* by :func:`build_advanced_group`, which is why + this cannot be the plain ``TranslatableMixin`` the parameter used to + name — that class has none of them. + """ + + _stun_edit: Any + _turn_edit: Any + _turn_user_edit: Any + _turn_cred_edit: Any + _hw_codec_combo: Any + + def _tr(self, widget: QWidget, key: str, setter: str = "") -> QWidget: + """Register ``widget`` for live re-translation and return it.""" + + def _on_hw_codec_changed(self) -> None: + """React to the hardware-codec selection changing.""" + + +def build_advanced_group(panel: AdvancedGroupHost, + include_hw_codec: bool = False) -> QGroupBox: + """Shared 'Advanced' STUN/TURN (+ optional hw codec) group.""" + group = panel._tr(QGroupBox(), "rd_webrtc_advanced_group") + grid = QGridLayout() + grid.addWidget(panel._tr(QLabel(), "rd_webrtc_stun_label"), 0, 0) + panel._stun_edit = QLineEdit(DEFAULT_STUN) + grid.addWidget(panel._stun_edit, 0, 1, 1, 3) + grid.addWidget(panel._tr(QLabel(), "rd_webrtc_turn_label"), 1, 0) + panel._turn_edit = panel._tr(QLineEdit(), "rd_webrtc_turn_placeholder") + grid.addWidget(panel._turn_edit, 1, 1, 1, 3) + grid.addWidget(panel._tr(QLabel(), "rd_webrtc_turn_user_label"), 2, 0) + panel._turn_user_edit = QLineEdit() + grid.addWidget(panel._turn_user_edit, 2, 1) + grid.addWidget(panel._tr(QLabel(), "rd_webrtc_turn_cred_label"), 2, 2) + panel._turn_cred_edit = QLineEdit() + panel._turn_cred_edit.setEchoMode(QLineEdit.EchoMode.Password) + grid.addWidget(panel._turn_cred_edit, 2, 3) + if include_hw_codec: + _add_hw_codec_row(panel, grid) + group.setLayout(grid) + return group + + +def _add_hw_codec_row(panel: AdvancedGroupHost, grid: QGridLayout) -> None: + """Add the hardware-codec picker, pre-selected to the active codec.""" + grid.addWidget(panel._tr(QLabel(), "rd_webrtc_hw_codec_label"), 3, 0) + combo = QComboBox() + panel._hw_codec_combo = combo + combo.addItem(_t("rd_webrtc_hw_codec_off"), "") + for name in available_hardware_codecs(): + combo.addItem(name, name) + active = active_hardware_codec() + if active: + index = combo.findData(active) + if index >= 0: + combo.setCurrentIndex(index) + combo.currentIndexChanged.connect(lambda _i: panel._on_hw_codec_changed()) + grid.addWidget(combo, 3, 1, 1, 3) + + +__all__ = ["DEFAULT_STUN", "AdvancedGroupHost", "build_advanced_group"] diff --git a/je_auto_control/gui/remote_desktop/webrtc_panel.py b/je_auto_control/gui/remote_desktop/webrtc_panel.py index b4849caf..93dc9c2f 100644 --- a/je_auto_control/gui/remote_desktop/webrtc_panel.py +++ b/je_auto_control/gui/remote_desktop/webrtc_panel.py @@ -12,7 +12,7 @@ from __future__ import annotations import logging -from typing import Optional +from typing import TYPE_CHECKING, Optional from PySide6.QtCore import QObject, Qt, QTimer, Signal from PySide6.QtGui import QImage @@ -27,6 +27,9 @@ from je_auto_control.gui.remote_desktop._helpers import ( _CollapsibleSection, _t, ) +from je_auto_control.gui.remote_desktop.advanced_group import ( + build_advanced_group, +) from je_auto_control.gui.remote_desktop.blanking_overlay import BlankingOverlay from je_auto_control.gui.remote_desktop.frame_display import _FrameDisplay from je_auto_control.gui.remote_desktop.remote_screen_window import ( @@ -51,7 +54,7 @@ from je_auto_control.utils.logging.logging_instance import autocontrol_logger from je_auto_control.utils.remote_desktop import ( MultiViewerHost, SessionRecorder, WebRTCConfig, WebRTCDesktopViewer, - active_hardware_codec, available_hardware_codecs, default_address_book, + default_address_book, default_trust_list, install_hardware_codec, is_webrtc_available, load_or_create_viewer_id, send_magic_packet, uninstall_hardware_codec, ) @@ -71,6 +74,14 @@ BANDWIDTH_PRESETS, fps_for_preset, ) +if TYPE_CHECKING: # imported lazily at runtime to keep startup cheap + from je_auto_control.utils.remote_desktop.file_sync import ( + FolderSyncEngine, + ) + from je_auto_control.utils.remote_desktop.lan_discovery import ( + HostAdvertiser, + ) + _DEFAULT_FPS = 24 _DEFAULT_MONITOR = 1 @@ -78,7 +89,6 @@ # to localhost without TLS, and operators put TLS in front via nginx / # Caddy. Hotspot S5332 acknowledged on a per-line basis; see callers. _DEFAULT_SIGNALING_URL = "http://127.0.0.1:8765" # NOSONAR python:S5332 -_DEFAULT_STUN = "stun:stun.l.google.com:19302" _QUALITY_DOT_STYLE = "background-color: #555; border-radius: 7px;" _JSON_FILE_FILTER = "JSON (*.json);;All (*)" @@ -118,43 +128,6 @@ class _PanelSignals(QObject): annotation = Signal(object) # dict -def _build_advanced_group(panel: TranslatableMixin, - include_hw_codec: bool = False) -> QGroupBox: - """Shared 'Advanced' STUN/TURN (+ optional hw codec) group.""" - group = panel._tr(QGroupBox(), "rd_webrtc_advanced_group") - grid = QGridLayout() - grid.addWidget(panel._tr(QLabel(), "rd_webrtc_stun_label"), 0, 0) - panel._stun_edit = QLineEdit(_DEFAULT_STUN) - grid.addWidget(panel._stun_edit, 0, 1, 1, 3) - grid.addWidget(panel._tr(QLabel(), "rd_webrtc_turn_label"), 1, 0) - panel._turn_edit = panel._tr(QLineEdit(), "rd_webrtc_turn_placeholder") - grid.addWidget(panel._turn_edit, 1, 1, 1, 3) - grid.addWidget(panel._tr(QLabel(), "rd_webrtc_turn_user_label"), 2, 0) - panel._turn_user_edit = QLineEdit() - grid.addWidget(panel._turn_user_edit, 2, 1) - grid.addWidget(panel._tr(QLabel(), "rd_webrtc_turn_cred_label"), 2, 2) - panel._turn_cred_edit = QLineEdit() - panel._turn_cred_edit.setEchoMode(QLineEdit.EchoMode.Password) - grid.addWidget(panel._turn_cred_edit, 2, 3) - if include_hw_codec: - grid.addWidget(panel._tr(QLabel(), "rd_webrtc_hw_codec_label"), 3, 0) - panel._hw_codec_combo = QComboBox() - panel._hw_codec_combo.addItem(_t("rd_webrtc_hw_codec_off"), "") - for name in available_hardware_codecs(): - panel._hw_codec_combo.addItem(name, name) - active = active_hardware_codec() - if active: - idx = panel._hw_codec_combo.findData(active) - if idx >= 0: - panel._hw_codec_combo.setCurrentIndex(idx) - panel._hw_codec_combo.currentIndexChanged.connect( - lambda _i: panel._on_hw_codec_changed(), - ) - grid.addWidget(panel._hw_codec_combo, 3, 1, 1, 3) - group.setLayout(grid) - return group - - def _checked_or(panel, attr: str, default: bool = False) -> bool: """Return ``panel..isChecked()`` if the widget exists, else default.""" widget = getattr(panel, attr, None) @@ -229,7 +202,7 @@ def __init__(self, parent: Optional[QWidget] = None) -> None: self._trust_list = default_trust_list() self._blanking: Optional[BlankingOverlay] = None self._viewer_screen_window: Optional[ViewerScreenWindow] = None - self._lan_advertiser = None + self._lan_advertiser: Optional["HostAdvertiser"] = None self._annotation_overlay: Optional[HostAnnotationOverlay] = None self._tray = install_host_tray( on_open=self._on_tray_open, @@ -255,7 +228,7 @@ def _build_ui(self) -> None: layout.addWidget(self._build_signaling_group()) layout.addWidget(self._build_config_group()) layout.addWidget(self._build_manual_group()) - layout.addWidget(_build_advanced_group(self, include_hw_codec=True)) + layout.addWidget(build_advanced_group(self, include_hw_codec=True)) layout.addWidget(self._build_trusted_group()) self._status_label = QLabel(_t("rd_webrtc_status_idle")) layout.addWidget(self._status_label) @@ -825,9 +798,16 @@ def _on_generate_offer(self) -> None: self._offer_view.setPlainText("") QTimer.singleShot(0, self._produce_offer) + def _require_multi_host(self) -> MultiViewerHost: + """Return the running host, or say the session is not up yet.""" + host = self._multi_host + if host is None: + raise RuntimeError(_t("rd_webrtc_not_started")) + return host + def _produce_offer(self) -> None: try: - session_id, offer = self._multi_host.create_session_offer() + session_id, offer = self._require_multi_host().create_session_offer() except (RuntimeError, OSError) as error: # PermissionError is an OSError self._show_error(error) return @@ -1044,8 +1024,9 @@ def _dispatch_session_menu(self, chosen, actions: dict, def _trust_session_viewer(self, sid: str) -> None: try: - with self._multi_host._lock: - host = self._multi_host._sessions.get(sid) + multi_host = self._require_multi_host() + with multi_host._lock: + host = multi_host._sessions.get(sid) full_vid = host.pending_viewer_id if host is not None else None if full_vid: self._trust_list.add(full_vid, label=f"sess {sid[:6]}") @@ -1311,13 +1292,13 @@ def __init__(self, parent: Optional[QWidget] = None) -> None: from je_auto_control.utils.remote_desktop import default_known_hosts self._known_hosts = default_known_hosts() try: - self._viewer_id = load_or_create_viewer_id() + self._viewer_id: Optional[str] = load_or_create_viewer_id() except OSError as error: autocontrol_logger.warning("viewer_id init: %r", error) self._viewer_id = None self._recorder: Optional[SessionRecorder] = None self._stats_poller: Optional[StatsPoller] = None - self._sync_engine = None + self._sync_engine: Optional["FolderSyncEngine"] = None self._auto_reconnect_attempts = 0 self._user_initiated_disconnect = False # AnyDesk-style pop-out: created on auth_ok, hidden on stop. @@ -1350,7 +1331,7 @@ def _build_ui(self) -> None: self._build_manual_group(), "rd_webrtc_manual_group", )) - layout.addWidget(_build_advanced_group(self)) + layout.addWidget(build_advanced_group(self)) layout.addWidget(self._wrap_collapsed( self._build_remote_files_group(), "rd_webrtc_files_group", @@ -1467,13 +1448,15 @@ def _on_toggle_sync(self, checked: bool) -> None: ) from pathlib import Path as _Path try: - self._sync_engine = FolderSyncEngine( + viewer = self._require_viewer() + engine = FolderSyncEngine( watch_dir=_Path(path), - sender=lambda local, name: self._viewer.send_file( + sender=lambda local, name: viewer.send_file( local, remote_name=name, ), ) - self._sync_engine.start() + self._sync_engine = engine + engine.start() except (RuntimeError, OSError) as error: # FileNotFoundError is an OSError QMessageBox.warning(self, "WebRTC", str(error)) self._sync_btn.setChecked(False) @@ -2254,7 +2237,7 @@ def _answer_and_push(self, offer_sdp: str) -> None: host_id = self._host_id_edit.text().strip() expected_dtls = self._known_hosts.dtls_fingerprint_for(host_id) if host_id else None try: - answer = self._viewer.process_offer( + answer = self._require_viewer().process_offer( offer_sdp, expected_dtls_fingerprint=expected_dtls, ) except (ValueError, RuntimeError, OSError) as error: @@ -2302,9 +2285,16 @@ def _on_create_answer(self) -> None: self._status_label.setText(_t("rd_webrtc_creating_answer")) QTimer.singleShot(0, lambda: self._produce_answer(offer)) + def _require_viewer(self) -> WebRTCDesktopViewer: + """Return the live viewer, or say it is not connected yet.""" + viewer = self._viewer + if viewer is None: + raise RuntimeError(_t("rd_webrtc_not_connected")) + return viewer + def _produce_answer(self, offer: str) -> None: try: - answer = self._viewer.process_offer(offer) + answer = self._require_viewer().process_offer(offer) except (ValueError, RuntimeError, OSError) as error: self._show_error(error) return diff --git a/test/verify/typing_contract_exempt.txt b/test/verify/typing_contract_exempt.txt index 7aa1dedb..0adc9b54 100644 --- a/test/verify/typing_contract_exempt.txt +++ b/test/verify/typing_contract_exempt.txt @@ -14,6 +14,5 @@ # "Does not type-check" means on at least one of the three targets mypy can be # pointed at (win32, linux, darwin) — not just the one CI happens to run on. # -# Measured entries: 1 +# Measured entries: 0 -je_auto_control.gui.remote_desktop.webrtc_panel