feat(connectors): hand source files to the document pipeline instead of extracting them (#6821)

* feat(connectors): hand source files to the document pipeline instead of extracting them

A connector that extracted text itself stranded the document on a second, weaker
parser. The shared pipeline routes PDFs to OCR — the only way a scanned page is
readable at all — and owns every other format's parser, but its OCR branch is
gated on `mimeType === 'application/pdf'` and connector documents were stored as
`text/plain`, so a connector PDF could never reach it. The same file dragged into
the UI was read by OCR; synced through a connector it got the local parser.

`ExternalDocument` can now carry the source file itself, and SharePoint and
OneDrive hand over anything the knowledge base can parse rather than extracting
it. The sync engine stores those bytes under the file's own name and type, so the
pipeline parses them exactly as it would an upload of the same file. Formats that
are already text stay on the text path: HTML still reduces to plain text and the
rest are UTF-8 decodes, so nothing already indexed changes representation.

The MIME type is derived from the extension rather than the source's own
declaration, so a provider that omits or mislabels it cannot strand a PDF on the
non-OCR path. Re-syncing an existing document now rewrites `mimeType` too, which
is what lets one stored as connector-extracted text stop declaring `text/plain`.

This removes the duplicate extraction path rather than leaving both in place:
`extractConnectorText` is text-only, and the guard against fabricated content
moves to the pipeline where parsing now happens. That guard still matters —
`DocParser` and `PptxParser` never throw, returning a placeholder sentence or
scraped archive bytes on a legacy binary or an image-only deck — so a `degraded`
result now fails the document with the same actionable message it produced
before, naming the modern container for legacy formats.

The in-flight byte budget already accounted for this: `estimateOpSizeBytes` reads
the true source size from listing metadata, so batching reserved against the real
file all along and merely over-reserved while only text was stored.

* fix(knowledge): guard every parser against empty output, not just the file parsers

Moving connector parsing into the pipeline exposed a gap on the OCR branch. OCR
reads a scanned page with no recoverable text as empty, and the empty-content
guard lived inside the file-parser path, so such a document chunked to nothing
and reported success — the same silently-complete-but-useless outcome the guard
exists to prevent. The check now sits above the parser choice and covers OCR
too.

Also preserves a source file's extension when its name is too long for a storage
key. The extension is what picks the parser; a truncated name would still parse
correctly by falling back to the display name, but only by luck.

* fix(knowledge): validate the stored artifact against the parser registry

Ten of the formats a connector now hands over — docm, dotx, xlsm, xlsb, xltx,
pptm, potx, odt, ods and odp — parse fine but are deliberately not offered as
upload types. `resolveStoredArtifactExtension` gated on the upload allowlist, so
it rejected every one of them and processing failed with `Unsupported file
type`. They worked before only because the connector extracted them itself and
stored the result as text.

The question the gate is asking is whether a parser can read the stored object,
which the parser registry answers; the upload allowlist answers a different
question about what we accept from a user.

Also matches the sibling comment style in the object literal it sits in, and
teaches two test mocks the newly imported symbol.

* fix(knowledge): carry the MIME type through hydration

A listing stub is built before the file is fetched and declares `text/plain` for
everything, so a hydrated PDF kept claiming plain text at the top level. Nothing
broke today only because storage reads `sourceFile.mimeType` — which is exactly
what makes it a trap: anything later reaching for `extDoc.mimeType`, the obvious
field, silently loses the OCR routing this change exists to restore.

The merge is now `mergeHydratedDocument` rather than an inline spread, so what
hydration must carry is a stated contract with a test behind it instead of a
literal that is easy to under-specify — which is how the field was missed.

Author: waleedlatif1

apps/sim/connectors/onedrive/onedrive.ts

@@ -6,14 +6,13 @@ import type { ConnectorConfig, ExternalDocument, ExternalDocumentList } from '@/
 import {
   CONNECTOR_MAX_FILE_BYTES,
   ConnectorFileTooLargeError,
-  ConnectorTextExtractionError,
   connectorFileExtension,
   extractConnectorText,
-  extractionFailedSkipReason,
   isIndexableConnectorFile,
   isSkippedDocument,
   markSkipped,
   parseTagDate,
+  pipelineParsedMimeType,
   readBodyWithLimit,
   sizeLimitSkipReason,
   stubOrSkipBySize,
@@ -103,13 +102,19 @@ async function downloadFileContent(accessToken: string, fileId: string): Promise
  * Fetches a file and extracts its indexable text — a UTF-8 decode for text
  * formats, and the shared knowledge-base parsers for Office documents and PDFs.
  */
-async function fetchFileContent(
+async function fetchFilePayload(
   accessToken: string,
   fileId: string,
   fileName: string
-): Promise<string> {
+): Promise<Pick<ExternalDocument, 'content' | 'sourceFile' | 'mimeType'>> {
   const buffer = await downloadFileContent(accessToken, fileId)
-  return extractConnectorText(buffer, fileName)
+
+  const mimeType = pipelineParsedMimeType(fileName)
+  if (mimeType) {
+    return { content: '', mimeType, sourceFile: { bytes: buffer, fileName, mimeType } }
+  }
+
+  return { content: extractConnectorText(buffer, fileName), mimeType: 'text/plain' }
 }
 
 /**
@@ -377,23 +382,16 @@ export const onedriveConnector: ConnectorConfig = {
     if (!item.file || !isIndexableConnectorFile(item.name)) return null
 
     try {
-      const content = await fetchFileContent(accessToken, item.id, item.name)
-      if (!content.trim()) return null
+      const payload = await fetchFilePayload(accessToken, item.id, item.name)
+      if (!payload.sourceFile && !payload.content.trim()) return null
 
       const stub = fileToStub(item)
-      return { ...stub, content, contentDeferred: false }
+      return { ...stub, ...payload, contentDeferred: false }
     } catch (error) {
       if (error instanceof ConnectorFileTooLargeError) {
         logger.info('Skipping oversized OneDrive file', { fileId: item.id, name: item.name })
         return markSkipped(fileToStub(item), sizeLimitSkipReason(error.limitBytes))
       }
-      if (error instanceof ConnectorTextExtractionError) {
-        logger.info('Skipping OneDrive file with no extractable text', {
-          fileId: item.id,
-          name: item.name,
-        })
-        return markSkipped(fileToStub(item), extractionFailedSkipReason(error.extension))
-      }
       /**
        * A transport or Graph failure that survived `fetchWithRetry`. Returning
        * `null` would drop the file from the run with no `failed` row and no error

apps/sim/connectors/sharepoint/sharepoint.test.ts

@@ -3,16 +3,12 @@
  */
 import { beforeEach, describe, expect, it, vi } from 'vitest'
 
-const { mockFetchWithRetry, mockParseBuffer } = vi.hoisted(() => ({
-  mockFetchWithRetry: vi.fn(),
-  mockParseBuffer: vi.fn(),
-}))
+const { mockFetchWithRetry } = vi.hoisted(() => ({ mockFetchWithRetry: vi.fn() }))
 
 vi.mock('@/lib/knowledge/documents/utils', () => ({
   fetchWithRetry: mockFetchWithRetry,
   VALIDATE_RETRY_OPTIONS: {},
 }))
-vi.mock('@/lib/file-parsers', () => ({ parseBuffer: mockParseBuffer }))
 vi.mock('@/components/icons', () => ({ MicrosoftSharepointIcon: () => null }))
 
 import {
@@ -496,45 +492,45 @@ describe('getDocument content extraction', () => {
     )
   }
 
-  it('indexes the parsed text of an Office document', async () => {
-    mockGraph({ ...itemRoute('f1', 'SOP.docx'), ...contentRoute('f1', 'ignored') })
-    mockParseBuffer.mockResolvedValue({
-      content: 'Approved vendor list',
-      metadata: { extractionMethod: 'mammoth' },
-    })
+  /**
+   * The connector hands an Office document over untouched so the shared pipeline
+   * parses it — the same path an upload of the same file takes, which is what
+   * routes PDFs through OCR.
+   */
+  it('delivers an Office document as its source file rather than extracting it', async () => {
+    mockGraph({ ...itemRoute('f1', 'SOP.docx'), ...contentRoute('f1', 'PK-docx-bytes') })
 
     const doc = await get('f1')
 
-    expect(doc?.content).toBe('Approved vendor list')
-    expect(doc?.skippedReason).toBeUndefined()
+    expect(doc?.content).toBe('')
+    expect(doc?.sourceFile?.fileName).toBe('SOP.docx')
+    expect(doc?.sourceFile?.mimeType).toBe(
+      'application/vnd.openxmlformats-officedocument.wordprocessingml.document'
+    )
+    expect(doc?.sourceFile?.bytes.toString()).toBe('PK-docx-bytes')
+    expect(doc?.mimeType).toBe(
+      'application/vnd.openxmlformats-officedocument.wordprocessingml.document'
+    )
     expect(doc?.contentDeferred).toBe(false)
   })
 
-  /**
-   * A degraded extraction must become a visible `failed` row, not a silent drop
-   * and not indexed placeholder text — the same treatment oversized files get.
-   */
-  it('surfaces a degraded extraction as a skipped document with an actionable reason', async () => {
-    mockGraph({ ...itemRoute('f2', 'Deck.ppt'), ...contentRoute('f2', 'ole2') })
-    mockParseBuffer.mockResolvedValue({
-      content: 'Unable to extract text from PowerPoint file.',
-      metadata: { extractionMethod: 'fallback', degraded: true },
-    })
+  it('declares a PDF as application/pdf so the pipeline can route it to OCR', async () => {
+    mockGraph({ ...itemRoute('f4', 'Contract.pdf'), ...contentRoute('f4', '%PDF-1.7 bytes') })
 
-    const doc = await get('f2')
+    const doc = await get('f4')
 
-    expect(doc?.content).toBe('')
-    expect(doc?.skippedReason).toContain('PPTX')
-    expect(doc?.externalId).toBe('f2')
+    expect(doc?.mimeType).toBe('application/pdf')
+    expect(doc?.sourceFile?.mimeType).toBe('application/pdf')
   })
 
-  it('reads a text file without invoking a parser', async () => {
+  it('still extracts a text file itself, since there is nothing for a parser to do', async () => {
     mockGraph({ ...itemRoute('f3', 'notes.txt'), ...contentRoute('f3', 'plain notes') })
 
     const doc = await get('f3')
 
     expect(doc?.content).toBe('plain notes')
-    expect(mockParseBuffer).not.toHaveBeenCalled()
+    expect(doc?.sourceFile).toBeUndefined()
+    expect(doc?.mimeType).toBe('text/plain')
   })
 })
 

apps/sim/connectors/sharepoint/sharepoint.ts

@@ -6,14 +6,13 @@ import type { ConnectorConfig, ExternalDocument, ExternalDocumentList } from '@/
 import {
   CONNECTOR_MAX_FILE_BYTES,
   ConnectorFileTooLargeError,
-  ConnectorTextExtractionError,
   connectorFileExtension,
   extractConnectorText,
-  extractionFailedSkipReason,
   isIndexableConnectorFile,
   isSkippedDocument,
   markSkipped,
   parseTagDate,
+  pipelineParsedMimeType,
   readBodyWithLimit,
   sizeLimitSkipReason,
   stubOrSkipBySize,
@@ -214,14 +213,20 @@ async function downloadFileContent(
  * Fetches a file and extracts its indexable text — a UTF-8 decode for text
  * formats, and the shared knowledge-base parsers for Office documents and PDFs.
  */
-async function fetchFileContent(
+async function fetchFilePayload(
   accessToken: string,
   driveId: string,
   itemId: string,
   fileName: string
-): Promise<string> {
+): Promise<Pick<ExternalDocument, 'content' | 'sourceFile' | 'mimeType'>> {
   const buffer = await downloadFileContent(accessToken, driveId, itemId, fileName)
-  return extractConnectorText(buffer, fileName)
+
+  const mimeType = pipelineParsedMimeType(fileName)
+  if (mimeType) {
+    return { content: '', mimeType, sourceFile: { bytes: buffer, fileName, mimeType } }
+  }
+
+  return { content: extractConnectorText(buffer, fileName), mimeType: 'text/plain' }
 }
 
 /**
@@ -925,11 +930,11 @@ export const sharepointConnector: ConnectorConfig = {
     }
 
     try {
-      const content = await fetchFileContent(accessToken, driveId, item.id, item.name)
-      if (!content.trim()) return null
+      const payload = await fetchFilePayload(accessToken, driveId, item.id, item.name)
+      if (!payload.sourceFile && !payload.content.trim()) return null
 
       const stub = itemToStub(item, siteName ?? siteUrl)
-      return { ...stub, content, contentDeferred: false }
+      return { ...stub, ...payload, contentDeferred: false }
     } catch (error) {
       if (error instanceof ConnectorFileTooLargeError) {
         logger.info('Skipping oversized SharePoint file', { fileId: item.id, name: item.name })
@@ -938,16 +943,6 @@ export const sharepointConnector: ConnectorConfig = {
           sizeLimitSkipReason(error.limitBytes)
         )
       }
-      if (error instanceof ConnectorTextExtractionError) {
-        logger.info('Skipping SharePoint file with no extractable text', {
-          fileId: item.id,
-          name: item.name,
-        })
-        return markSkipped(
-          itemToStub(item, siteName ?? siteUrl),
-          extractionFailedSkipReason(error.extension)
-        )
-      }
       /**
        * A transport or Graph failure that survived `fetchWithRetry`. Returning
        * `null` would drop the file from the run with no `failed` row and no error

apps/sim/connectors/types.ts

@@ -29,10 +29,29 @@ export interface ExternalDocument {
   externalId: string
   /** Document title / filename */
   title: string
-  /** Extracted text content */
+  /** Extracted text content. Empty when {@link ExternalDocument.sourceFile} carries the document instead. */
   content: string
   /** MIME type of the content */
   mimeType: string
+  /**
+   * The source file itself, for connectors that hand over the original document
+   * rather than text they extracted from it.
+   *
+   * Preferred for any format the knowledge base can parse. Extracting inside a
+   * connector strands the document on a second, weaker parser: the shared
+   * pipeline routes PDFs to OCR (so scanned pages are readable at all) and owns
+   * every other format's parser, while a connector doing its own extraction
+   * stores plain text that no longer declares what it came from.
+   *
+   * Carried as one object so the bytes can never disagree with the name and type
+   * that describe them.
+   */
+  sourceFile?: {
+    bytes: Buffer
+    /** Name whose extension names the format, e.g. `Report.pdf`. */
+    fileName: string
+    mimeType: string
+  }
   /** Link back to the original document */
   sourceUrl?: string
   /** Hash of content for change detection (format varies by connector) */