Add Azure Blob Storage as a File Sync source

[paullizer]

Summary

Add Azure Blob Storage as a first-class File Sync source alongside Azure Files. Administrators and authorized workspace users should be able to configure a storage account, container, and optional blob-name prefix, then ingest new and changed blobs through the existing File Sync pipeline.

User Value

Many organizations store source documents in Blob Storage rather than an Azure Files share. Supporting Blob Storage lets those repositories participate in the same scheduled ingestion, document processing, tagging, and workspace experiences without requiring users to copy content into Azure Files first.

Proposed Behavior

• Add an Azure Blob Storage source type to the existing File Sync source registry and admin-controlled source-type visibility settings.
• Support the File Sync workspace scopes that are already available to other eligible source types, using the same source ownership and workspace authorization checks.
• Collect a storage account URL or account name, container name, and optional blob-name prefix. Normalize and validate these values server-side.
• Reuse the existing workspace identity and Key Vault-backed File Sync credential models. Prefer managed identity and least-privilege Azure RBAC; support other credential modes only where they can be stored and resolved without exposing secrets to the browser or Cosmos DB records.
• Provide a connection test that verifies authentication, container access, and prefix validity without downloading the full source.
• Enumerate blobs beneath the configured prefix, treating blob names as virtual paths and ignoring directory-marker records where appropriate.
• Use stable source item identifiers plus Blob properties such as ETag, last-modified time, and content length to detect additions and changes efficiently across runs.
• Stream blob content into the existing File Sync ingestion path so file-size limits, supported-format checks, document processing, tags, source attribution, run metrics, and retry behavior remain consistent with other connectors.
• Apply existing File Sync deletion and rename policies when previously synchronized blobs disappear or move.
• Redact credentials and sensitive endpoint details from API responses, logs, run errors, and frontend state.
• Document the required Azure role assignments, recommended Storage Blob Data Reader scope, supported credential modes, network/private-endpoint considerations, and container/prefix configuration.

Acceptance Criteria

• Azure Blob Storage can be enabled or hidden through the existing admin-managed File Sync source-type configuration.
• Authorized users can create, edit, test, enable, disable, run, and delete a Blob Storage source through the same File Sync workflows used by other source types.
• Source creation and update validate the account endpoint, container, prefix, authentication mode, and referenced workspace identity on the server.
• Personal, group, and public workspace authorization boundaries remain enforced; a user cannot test or run a source outside their authorized workspace scope.
• Managed identity authentication works with least-privilege Blob data access. Any supported client secret or connection string is stored through the existing Key Vault-backed secret path and is never returned to the frontend.
• Connection testing distinguishes invalid credentials, missing RBAC, unreachable endpoints, missing containers, and invalid prefixes without leaking secret values.
• A sync run recursively discovers eligible blobs under the configured prefix and downloads them without loading large files entirely into application memory.
• New and changed blobs enter the existing ingestion pipeline with correct filename, virtual path, content type, size, last-modified value, source attribution, and configured path-tag behavior.
• A second unchanged run does not reprocess unchanged blobs, while ETag or relevant metadata changes cause the blob to be processed again.
• Deleted and renamed blobs follow the connector-independent File Sync reconciliation and deletion policies without affecting documents from another source.
• Source and item identifiers remain stable and collision-safe across containers, prefixes, workspaces, and storage accounts.
• Failures are recorded in File Sync run/item status, are retryable where appropriate, and do not expose credentials or unauthorized endpoint details.
• Automated tests cover configuration validation, source visibility, all supported authentication paths, workspace isolation, prefix filtering, virtual paths, incremental change detection, deletion reconciliation, large-file streaming, and sanitized errors using mocked Azure SDK clients where practical.
• Functional or integration validation covers at least one real Blob container using managed identity or an equivalent CI-safe test environment.
• File Sync documentation explains setup, RBAC, networking, configuration, expected sync semantics, troubleshooting, and known limitations.

Notes

• Initial scope is inbound File Sync from Blob Storage into SimpleChat. Outbound synchronization, blob uploads, and mutation of source blobs are not part of this issue.
• Azure Data Lake Storage Gen2-specific ACL management and filesystem semantics are not required unless they work transparently through the same Blob API contract; they can be scoped separately if needed.
• Reuse the established connector, workspace identity, Key Vault, sanitization, and run/item reconciliation abstractions rather than adding a separate synchronization subsystem.
• Relevant implementation areas include functions_file_sync.py, route_backend_file_sync.py, workspace File Sync UI modules, source-type visibility settings, workspace identities, and Key Vault secret handling.