feat: web domain allowlist and markdown report rendering

docs/en/configuration/config-files.md

@@ -85,6 +85,9 @@ api_key = "sk-xxx"
 base_url = "https://api.pythinker.com/coding/v1/fetch"
 api_key = "sk-xxx"
 
+[web]
+allowed_domains = ["example.com", "docs.python.org"]
+
 [mcp.client]
 tool_call_timeout_ms = 60000
 ```
@@ -199,6 +202,16 @@ Configures web fetch service. When enabled, the `FetchURL` tool prioritizes usin
 When configuring the Pythinker platform using the `/login` command, search and fetch services are automatically configured.
 :::
 
+### `web`
+
+`web` configures policy shared by the `FetchURL` and `SearchWeb` tools.
+
+| Field | Type | Default | Description |
+| --- | --- | --- | --- |
+| `allowed_domains` | `array<string>` | _unset_ | When set, web fetch and search may only touch these domains and their subdomains. `FetchURL` rejects URLs on other hosts before making any request — including redirect targets, which are re-validated on every hop — and `SearchWeb` drops results from other domains. Unset or empty means unrestricted. Entries must be bare hostnames (e.g. `example.com`), not URLs, paths, or `host:port`. |
+
+This is a coarse governance control layered on top of the existing SSRF protections (which always block private, loopback, link-local, multicast, and reserved addresses); it does not replace them. Matching is label-aware: `example.com` matches `example.com` and `docs.example.com`, but not `notexample.com`.
+
 ### `mcp`
 
 `mcp` configures MCP client behavior.

src/pythinker_code/agents/default/system.md

@@ -188,7 +188,7 @@ The operating environment is not in a sandbox. Any actions you do will immediate
 
 ## Date and Time
 
-The current date and time in ISO format is `${PYTHINKER_NOW}`. This is only a reference for you when searching the web, or checking file modification time, etc. If you need the exact time, use Shell tool with proper command.
+The current date and time in ISO format is `${PYTHINKER_NOW}`. Treat this as the authoritative present — it reflects the real "now", which is later than your training data suggests. Anchor all reasoning about the current date, the year, recency, and what counts as the "latest" version or release to `${PYTHINKER_NOW}`; do not fall back on an earlier year you might assume from training. Use it as your reference when searching the web or checking file modification times. If you need the exact time, use the Shell tool with a proper command.
 
 ## Working Directory
 
@@ -261,6 +261,15 @@ Identify the skills that are likely to be useful for the tasks you are currently
 
 Only read skill details when needed to conserve the context window.
 
+# Output Formatting
+
+Your responses are rendered as Markdown in a terminal. Emit well-formed Markdown so it renders cleanly:
+
+- **Tables:** put the header row on its own line, the `|---|---|` delimiter row on the immediately following line (no blank line between them), and one row per line. Never glue a table onto adjacent prose (e.g. `Findings| Col |`) and never cram multiple rows onto one line. Leave a blank line before and after the table.
+- Prefer a short bullet list over a table when there are only a few items or any cell is long; reserve tables for genuinely tabular data with short cells.
+- **Code fences are for code only.** Use triple-backtick blocks tagged with a language (for example, `python` or `toml`) solely for source, config, or commands — one snippet per block. Never wrap a prose report, finding list, checklist, or ASCII box in a fence to align or frame it; write it as normal Markdown (headings, bullets, tables) so it renders cleanly.
+- **Status icons sparingly.** A check/cross/dot can mark a single headline result, but do not prefix every line with one. Use plain words for severity and outcomes (e.g. `High`, `PASS`, `0 findings`). The terminal renders icons as calm monochrome glyphs only outside code fences — another reason not to box reports.
+
 # Ultimate Reminders
 
 At any time, you should be HELPFUL, CONCISE, and ACCURATE. Be thorough in your actions — test what you build, verify what you change — not in your explanations.

src/pythinker_code/config.py

@@ -14,6 +14,7 @@
     SecretStr,
     ValidationError,
     field_serializer,
+    field_validator,
     model_validator,
 )
 from tomlkit.exceptions import TOMLKitError
@@ -217,6 +218,41 @@ class Services(BaseModel):
     """Pythinker AI Fetch configuration."""
 
 
+class WebConfig(BaseModel):
+    """Web fetch/search policy."""
+
+    allowed_domains: list[str] | None = Field(
+        default=None,
+        description=(
+            "If set, web fetch and search may only touch these domains and their "
+            "subdomains. None or empty means unrestricted (default)."
+        ),
+    )
+
+    @field_validator("allowed_domains")
+    @classmethod
+    def _validate_allowed_domains(cls, value: list[str] | None) -> list[str] | None:
+        for entry in value or []:
+            cleaned = entry.strip()
+            if not cleaned:
+                raise ValueError(
+                    "Invalid allowed_domains entry: empty or whitespace-only hostname. "
+                    "Remove it, or omit allowed_domains entirely to leave web access "
+                    "unrestricted."
+                )
+            if cleaned.strip(".") == "":
+                raise ValueError(
+                    f"Invalid allowed_domains entry {entry!r}: hostname must contain "
+                    "domain labels, not only dots."
+                )
+            if any(char.isspace() for char in cleaned) or any(char in cleaned for char in "/:"):
+                raise ValueError(
+                    f"Invalid allowed_domains entry {entry!r}: use a bare hostname "
+                    "like 'example.com', not a URL, path, or host:port."
+                )
+        return value
+
+
 class FeedbackConfig(BaseModel):
     """User-submitted feedback endpoint configuration."""
 
@@ -364,6 +400,7 @@ class Config(BaseModel):
     )
     services: Services = Field(default_factory=Services, description="Services configuration")
     memory: MemoryConfig = Field(default_factory=MemoryConfig, description="Memory configuration")
+    web: WebConfig = Field(default_factory=WebConfig, description="Web fetch/search policy")
     feedback: FeedbackConfig = Field(
         default_factory=FeedbackConfig,
         description="User-submitted feedback endpoint configuration",

src/pythinker_code/tools/web/_allowlist.py

@@ -0,0 +1,27 @@
+"""Domain allowlist matching shared by the web fetch and search tools."""
+
+from __future__ import annotations
+
+
+def _normalize(entry: str) -> str:
+    return entry.strip().strip(".").lower()
+
+
+def host_in_allowlist(host: str | None, allowed: list[str] | None) -> bool:
+    """Return whether *host* is permitted by the *allowed* domain list.
+
+    A ``None`` or empty allowlist imposes no restriction (returns ``True``),
+    preserving unconfigured behavior. Otherwise a host matches when it equals an
+    allowlist entry or is a subdomain of one. Matching is label-aware and
+    case-insensitive: ``example.com`` matches ``example.com`` and
+    ``docs.example.com`` but not ``notexample.com``.
+    """
+    entries = [normalized for entry in (allowed or []) if (normalized := _normalize(entry))]
+    if not entries:
+        return True
+
+    host = (host or "").strip().rstrip(".").lower()
+    if not host:
+        return False
+
+    return any(host == entry or host.endswith(f".{entry}") for entry in entries)

src/pythinker_code/tools/web/fetch.md

@@ -1 +1 @@
-Fetch a web page from a URL and extract main text content from it.
+Fetch a web page from a URL and extract main text content from it. Requests may be restricted to a configured set of allowed domains; fetching a disallowed host (including via a redirect) returns an error.

src/pythinker_code/tools/web/fetch.py

@@ -15,6 +15,7 @@
 from pythinker_code.soul.agent import Runtime
 from pythinker_code.soul.toolset import get_current_tool_call_or_none
 from pythinker_code.tools.utils import ToolResultBuilder, load_desc
+from pythinker_code.tools.web._allowlist import host_in_allowlist
 from pythinker_code.utils.aiohttp import new_client_session
 from pythinker_code.utils.logging import logger
 
@@ -23,13 +24,16 @@
 _REDIRECT_STATUSES = frozenset({301, 302, 303, 307, 308})
 
 
-def _validate_fetch_url(url: str) -> str | None:
+def _validate_fetch_url(url: str, allowed_domains: list[str] | None = None) -> str | None:
     parsed = urlparse(url)
     if parsed.scheme not in {"http", "https"}:
         return "Only http and https URLs are supported."
     if not parsed.hostname:
         return "URL must include a host."
 
+    if not host_in_allowlist(parsed.hostname, allowed_domains):
+        return "URL host is not in the configured web allowlist."
+
     try:
         infos = socket.getaddrinfo(parsed.hostname, parsed.port, type=socket.SOCK_STREAM)
     except socket.gaierror:
@@ -76,23 +80,29 @@ def __init__(self, reason: str) -> None:
 
 
 async def _get_revalidating_redirects(
-    session: aiohttp.ClientSession, url: str, headers: dict[str, str]
+    session: aiohttp.ClientSession,
+    url: str,
+    headers: dict[str, str],
+    allowed_domains: list[str] | None = None,
 ) -> aiohttp.ClientResponse:
     """GET ``url``, following redirects manually and re-validating every hop.
 
     aiohttp follows redirects internally without re-checking the destination, so
     a public URL that 30x-redirects to a private/link-local address (e.g. a cloud
     metadata endpoint at 169.254.169.254) would otherwise sail past
     ``_validate_fetch_url``. We disable automatic redirects and validate each
-    ``Location`` before following it.
+    ``Location`` against both the SSRF guard and the configured domain allowlist
+    before following it.
 
     Returns the final, open, non-redirect response (the caller owns closing it).
     Raises ``_FetchBlocked`` if any hop is blocked or the redirect limit is hit.
     """
     current = url
-    for _ in range(MAX_FETCH_REDIRECTS + 1):
-        if reason := _validate_fetch_url(current):
-            raise _FetchBlocked(reason)
+    for hop in range(MAX_FETCH_REDIRECTS + 1):
+        if reason := _validate_fetch_url(current, allowed_domains):
+            if hop == 0:
+                raise _FetchBlocked(reason)
+            raise _FetchBlocked(f"redirect to a disallowed location: {reason}")
         response = await session.get(current, headers=headers, allow_redirects=False)
         location = response.headers.get(aiohttp.hdrs.LOCATION)
         if response.status in _REDIRECT_STATUSES and location:
@@ -116,6 +126,7 @@ def __init__(self, config: Config, runtime: Runtime):
         super().__init__()
         self._runtime = runtime
         self._service_config = config.services.pythinker_ai_fetch
+        self._allowed_domains = config.web.allowed_domains
 
     @override
     async def __call__(self, params: Params) -> ToolReturnValue:
@@ -134,11 +145,18 @@ async def __call__(self, params: Params) -> ToolReturnValue:
                 return ret
             logger.warning("Failed to fetch URL via service: {error}", error=ret.message)
             # fallback to local fetch if service fetch fails
-        return await self.fetch_with_http_get(params)
+        return await self.fetch_with_http_get(params, self._allowed_domains)
 
     @staticmethod
-    async def fetch_with_http_get(params: Params) -> ToolReturnValue:
+    async def fetch_with_http_get(
+        params: Params, allowed_domains: list[str] | None = None
+    ) -> ToolReturnValue:
         builder = ToolResultBuilder(max_line_length=None)
+        # Validate the initial URL up front so a disallowed host is rejected
+        # before any network session is opened; the redirect helper below
+        # re-validates every subsequent hop.
+        if reason := _validate_fetch_url(params.url, allowed_domains):
+            return builder.error(f"Failed to fetch URL: {reason}", brief="URL blocked")
         headers = {
             "User-Agent": (
                 "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 "
@@ -150,7 +168,9 @@ async def fetch_with_http_get(params: Params) -> ToolReturnValue:
             fetch_timeout = aiohttp.ClientTimeout(total=180, sock_read=60, sock_connect=15)
             async with new_client_session(timeout=fetch_timeout) as session:
                 try:
-                    response = await _get_revalidating_redirects(session, params.url, headers)
+                    response = await _get_revalidating_redirects(
+                        session, params.url, headers, allowed_domains
+                    )
                 except _FetchBlocked as blocked:
                     return builder.error(
                         f"Failed to fetch URL: {blocked.reason}", brief="URL blocked"
@@ -245,7 +265,7 @@ async def _fetch_with_service(self, params: Params) -> ToolReturnValue:
                 "Fetch service is not configured. You may want to try other methods to fetch.",
                 brief="Fetch service not configured",
             )
-        if reason := _validate_fetch_url(params.url):
+        if reason := _validate_fetch_url(params.url, self._allowed_domains):
             return builder.error(f"Failed to fetch URL: {reason}", brief="URL blocked")
 
         headers = {

src/pythinker_code/tools/web/search.md

@@ -1 +1 @@
-WebSearch tool allows you to search on the internet to get latest information, including news, documents, release notes, blog posts, papers, etc.
+WebSearch tool allows you to search on the internet to get latest information, including news, documents, release notes, blog posts, papers, etc. Results may be limited to a configured set of allowed domains.

src/pythinker_code/tools/web/search.py

@@ -1,5 +1,6 @@
 from pathlib import Path
 from typing import override
+from urllib.parse import urlparse
 
 import aiohttp
 from pydantic import BaseModel, Field, ValidationError
@@ -12,6 +13,7 @@
 from pythinker_code.soul.toolset import get_current_tool_call_or_none
 from pythinker_code.tools import SkipThisTool
 from pythinker_code.tools.utils import ToolResultBuilder, load_desc
+from pythinker_code.tools.web._allowlist import host_in_allowlist
 from pythinker_code.utils.aiohttp import new_client_session
 from pythinker_code.utils.logging import logger
 
@@ -53,6 +55,7 @@ def __init__(self, config: Config, runtime: Runtime):
         self._api_key = config.services.pythinker_ai_search.api_key
         self._oauth_ref = config.services.pythinker_ai_search.oauth
         self._custom_headers = config.services.pythinker_ai_search.custom_headers or {}
+        self._allowed_domains = config.web.allowed_domains
 
     @override
     async def __call__(self, params: Params) -> ToolReturnValue:
@@ -145,6 +148,26 @@ async def __call__(self, params: Params) -> ToolReturnValue:
                 brief="Search request failed",
             )
 
+        if self._allowed_domains:
+            kept = [
+                result
+                for result in results
+                if host_in_allowlist(urlparse(result.url).hostname, self._allowed_domains)
+            ]
+            dropped = len(results) - len(kept)
+            results = kept
+            if dropped:
+                builder.extras(allowlist_filtered=dropped)
+            if not results:
+                # Structured zero-result signal so the renderer reports "0
+                # results" instead of misreading the prose below as one result.
+                builder.extras(returned_results=0)
+                return builder.ok(
+                    f"All {dropped} search result(s) were outside the configured "
+                    "web allowlist and have been omitted.",
+                    brief="Filtered by allowlist",
+                )
+
         for i, result in enumerate(results):
             if i > 0:
                 builder.write("---\n\n")