Skip to content

Commit 31521d6

Browse files
authored
fix(connectors): validate and repair the knowledge-base connector fleet (#6757)
* fix(connectors): validate and repair all 61 knowledge-base connectors Audits every KB connector against its provider's live API documentation and fixes what the audit found. The dominant defect class is deletion reconciliation: the sync engine hard-deletes any stored document absent from a "full" listing, and most connectors had a path where a truncated or errored listing failed to set `syncContext.listingCapped`. Highest-impact fixes: - linear: `getDocument` was dead code. The query declared `$id: ID!` where the schema is `issue(id: String!)`, so every call failed variable validation. - salesforce: `v62.0` was substituted into a `{version}` template that already contains the `v`, so every REST call 404'd. SOQL `LIMIT` was also used as a page size, silently capping every sync at 200 records. - notion: only the first level of blocks was fetched, so tables indexed empty. - microsoft-teams: `/messages` returns messages without replies, so no threaded content was ever indexed. - gmail: an empty page discarded `nextPageToken`, which reads as a complete empty listing and hard-deletes every stored thread. - confluence: the CQL path paginated with `start` and `totalSize`, neither of which exists on that endpoint, so label-filtered syncs stopped after one page. - box: `supportsRefreshTokenRotation` was unset, so Box's rotated refresh token was discarded and every credential died on its second refresh. Removes the Evernote integration entirely: the classic EDAM API is deprecated, its sandbox is decommissioned, and developer tokens are no longer obtainable. Makes SFTP host-key verification mandatory, and adds an attendee-PII opt-out to google-calendar and google-meet (default on, so existing sources are unchanged). Bumps the contentHash namespace for notion, google-docs and hubspot so existing documents re-hydrate once and actually receive the content fixes above. * fix(connectors): close swallow-into-empty and cursor-taper regressions Ship-gate pass over the connector audit. Every finding was re-verified against the provider's live documentation or machine-readable spec before being acted on; several pass-3 edits were reverted rather than extended. Correctness fixes: - fireflies: a 2xx with an unparseable body returned an empty listing instead of throwing. fireflies runs a full sync every time, so a fault persisting across two syncs would have tombstoned all indexed docs. - linear: same shape via `data.issues || {}` on a non-nullable connection. - greenhouse: a 403 from a key without scorecard permission was treated as transient, appending `:partial` to the hash. That never matches the list stub, forcing full re-hydration of every candidate on every sync forever. - google-meet: `fetchParticipants` carried a 404 swallow copied from its transcript siblings, freezing every speaker as "Unknown". - airtable, asana, ashby: reverted page-size tapers applied over opaque cursor tokens. The cap was already enforced server-side. - google-docs: response byte cap resolved to 800MB and could never fire. - google-forms, google-vault, notion, sharepoint, dropbox: `getDocument` now throws on transient failure instead of returning null, which the engine reads as absence. Security: - Retry headers are attached non-enumerably. TypeScript `private` is compile-time only, so `SecureFetchHeaders.setCookies` was an own enumerable property that the logger serialized into sync logs. Docs and dead code: - Corrected six fabricated doc citations (github, jira, jsm, linear, google-meet, dropbox) and removed the Evernote integration entirely. * refactor(jira): type the ADF node helpers with unknown instead of any * fix(connectors): throw on misconfigured typeform/zendesk sources instead of returning null A null from getDocument reads as documented absence, so on an add the document is dropped with neither a failure counter nor a log. Both listDocuments paths already throw on the same missing config. * fix(connectors): keep confluence CQL page size constant; unify monday API version The CQL search endpoint paginates by opaque cursor, and Atlassian does not document that a cursor issued against one limit survives a request asking for a different one. Narrowing limit to the remaining budget was the same pattern reverted on airtable, asana, and ashby. The page size is now constant and the cap is applied by trimming the returned page, which keeps the cap exact without varying the request. Monday OAuth getUserInfo hardcoded API-Version 2024-10 while every other monday surface reads MONDAY_API_VERSION, defeating the single-source pin. * fix(connectors): act on the final validation sweep Findings from a read-only /validate-connector pass over all 59 changed connectors, verified against provider specs before acting. Silent-drop fixes (a fulfilled null from getDocument records no failure and no log, so the document vanishes): - ashby: candidate.info returning success with an unusable payload. Ashby sets contentDeferred, so this path is live. - azure-devops: an unresolvable branch, likewise live. - dropbox: 409 covers the whole LookupError union, and restricted_content and locked both mean the file still exists. Only not_found is absence. - docusign: fetchFormValues swallowed every non-OK status, baking a permanently incomplete document since the hash is metadata-only. typeform: 'all' sent response_type=started,partial,completed, but Typeform documents only partial and completed. An unknown enum member risks a 400 that fails the whole sync, and staging omitted the parameter entirely, so this shipped as a regression. Now requests the widest documented set. github: removes a utf-8 blob branch justified by a misattributed quote — that sentence describes the encoding REQUEST parameter of Create a blob; the GET response is documented as always base64. Also corrects two comments that hid a real drop: >1 MB files under vnd.github+json 403 rather than returning encoding: none. hubspot: routes HTML detection through a shared anchored helper. The loose pattern matched angle-bracketed prose such as an email address, and htmlToPlainText deletes the span and collapses line structure. This matters now because the hubspot:v2: bump rewrites every live document once. youtube: drops an invented channel-ID format quote. * fix(connectors): converge incidentio partial hash on settled statuses A 403 from a key without incident_updates permission, or a 404, returns on every sync. Marking those incomplete appended :partial to a hash that then never matched the listing stub, so the incident re-hydrated forever without converging. Only a transient failure may mark content incomplete now, which matches how greenhouse already treats the same class. * fix(connectors): flag WIQL truncation unconditionally; stop swallowing docusign form-data failures azure-devops: the 20,000-item WIQL ceiling was probed by asking for a matching item with an id beyond the largest returned. That probe is unsound — buildWiql orders by ChangedDate DESC while ids are assigned in creation order, so the highest-id match is almost always inside the returned window. The probe came back empty for genuinely truncated projects, left the listing unflagged, and let deletion reconciliation remove every indexed item outside it. Flag unconditionally instead; the cost is a project sitting exactly at the ceiling not reconciling deletions until a full resync. docusign: fetchFormValues threw on a non-404 status and then caught its own throw, returning []. The earlier fix was a no-op. The catch now rethrows, so a transient failure produces a failed row instead of a permanently incomplete document under a metadata-only hash. * fix(connectors): restore airtable AI Text indexing; floor sentry maxIssues airtable: staging rendered object cells with a JSON.stringify fallback, so AI Text values and nested lookup arrays reached the index. This branch replaced that with a fixed key-probe list to stop attachment-URL hash churn, but the probe list has no fallback — aiText ({state,isStale,value}) and nested arrays rendered to the empty string and vanished from every document, and the content-derived hash never moved when the text regenerated. Read `value` last, after the existing probes, and recurse on nested arrays. The generated text is stable rather than an expiring signed URL, so this does not reintroduce churn. sentry: maxIssues now feeds the request limit, and Sentry rejects a non-integer. validateConfig accepts a fractional entry, so a config that saved cleanly would fail every sync at listing time. Staging was immune only because it sent a hardcoded page size.
1 parent 557a474 commit 31521d6

203 files changed

Lines changed: 13633 additions & 8096 deletions

File tree

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

apps/docs/components/icons.tsx

Lines changed: 0 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -2769,14 +2769,6 @@ export function ExtendIcon(props: SVGProps<SVGSVGElement>) {
27692769
)
27702770
}
27712771

2772-
export function EvernoteIcon(props: SVGProps<SVGSVGElement>) {
2773-
return (
2774-
<svg {...props} viewBox='0 0 32 32' fill='#7fce2c' xmlns='http://www.w3.org/2000/svg'>
2775-
<path d='M29.343 16.818c.1 1.695-.08 3.368-.305 5.045-.225 1.712-.508 3.416-.964 5.084-.3 1.067-.673 2.1-1.202 3.074-.65 1.192-1.635 1.87-2.992 1.924l-3.832.036c-.636-.017-1.278-.146-1.9-.297-1.192-.3-1.862-1.1-2.06-2.3-.186-1.08-.173-2.187.04-3.264.252-1.23 1-1.96 2.234-2.103.817-.1 1.65-.077 2.476-.1.205-.007.275.098.203.287-.196.53-.236 1.07-.098 1.623.053.207-.023.307-.26.305a7.77 7.77 0 0 0-1.123.053c-.636.086-.96.47-.96 1.112 0 .205.026.416.066.622.103.507.45.78.944.837 1.123.127 2.247.138 3.37-.05.675-.114 1.08-.54 1.16-1.208.152-1.3.155-2.587-.228-3.845-.33-1.092-1.006-1.565-2.134-1.7l-3.36-.54c-1.06-.193-1.7-.887-1.92-1.9-.13-.572-.14-1.17-.214-1.757-.013-.106-.074-.208-.1-.3-.04.1-.106.212-.117.326-.066.68-.053 1.373-.185 2.04-.16.8-.404 1.566-.67 2.33-.185.535-.616.837-1.205.8a37.76 37.76 0 0 1-7.123-1.353l-.64-.207c-.927-.26-1.487-.903-1.74-1.787l-1-3.853-.74-4.3c-.115-.755-.2-1.523-.083-2.293.154-1.112.914-1.903 2.04-1.964l3.558-.062c.127 0 .254.003.373-.026a1.23 1.23 0 0 0 1.01-1.255l-.05-3.036c-.048-1.576.8-2.38 2.156-2.622a10.58 10.58 0 0 1 4.91.26c.933.275 1.467.923 1.715 1.83.058.22.146.3.37.287l2.582.01 3.333.37c.686.095 1.364.25 2.032.42 1.165.298 1.793 1.112 1.962 2.256l.357 3.355.3 5.577.01 2.277zm-4.534-1.155c-.02-.666-.07-1.267-.444-1.784a1.66 1.66 0 0 0-2.469-.15c-.364.4-.494.88-.564 1.4-.008.034.106.126.16.126l.8-.053c.768.007 1.523.113 2.25.393.066.026.136.04.265.077zM8.787 1.154a3.82 3.82 0 0 0-.278 1.592l.05 2.934c.005.357-.075.45-.433.45L5.1 6.156c-.583 0-1.143.1-1.554.278l5.2-5.332c.02.013.04.033.06.053z' />
2776-
</svg>
2777-
)
2778-
}
2779-
27802772
export function ElevenLabsIcon(props: SVGProps<SVGSVGElement>) {
27812773
return (
27822774
<svg

apps/docs/components/ui/icon-mapping.ts

Lines changed: 0 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -72,7 +72,6 @@ import {
7272
EnrichmentIcon,
7373
EnrichSoIcon,
7474
EnrowIcon,
75-
EvernoteIcon,
7675
ExaAIIcon,
7776
ExtendIcon,
7877
EyeIcon,
@@ -334,7 +333,6 @@ export const blockTypeToIconMap: Record<string, IconComponent> = {
334333
enrich: EnrichSoIcon,
335334
enrichment: EnrichmentIcon,
336335
enrow: EnrowIcon,
337-
evernote: EvernoteIcon,
338336
exa: ExaAIIcon,
339337
extend: ExtendIcon,
340338
extend_v2: ExtendIcon,

apps/docs/content/docs/en/integrations/evernote.mdx

Lines changed: 0 additions & 282 deletions
This file was deleted.

apps/docs/content/docs/en/integrations/meta.json

Lines changed: 0 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -73,7 +73,6 @@
7373
"enrich",
7474
"enrichment",
7575
"enrow",
76-
"evernote",
7776
"exa",
7877
"extend",
7978
"fathom",

apps/docs/content/docs/en/knowledgebase/connectors.mdx

Lines changed: 4 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -12,9 +12,9 @@ Connectors continuously sync documents from external services into your knowledg
1212

1313
## Available Connectors
1414

15-
<Image src="/static/connectors/connectors-sources.png" alt="Connect Source picker showing a searchable list of available connectors including Airtable, Asana, Confluence, Discord, Dropbox, Evernote, Fireflies, GitHub, and Gmail" width={800} height={500} />
15+
<Image src="/static/connectors/connectors-sources.png" alt="Connect Source picker showing a searchable list of available connectors including Airtable, Asana, Confluence, Discord, Dropbox, Fireflies, GitHub, and Gmail" width={800} height={500} />
1616

17-
Sim ships with 61 built-in connectors:
17+
Sim ships with 60 built-in connectors:
1818

1919
| Category | Connectors |
2020
|----------|-----------|
@@ -28,7 +28,7 @@ Sim ships with 61 built-in connectors:
2828
| **Support** | Intercom, ServiceNow, Zendesk, Zoho Desk |
2929
| **Incident Management** | incident.io, Rootly, PagerDuty |
3030
| **Data** | Airtable |
31-
| **Note-taking** | Evernote, Obsidian |
31+
| **Note-taking** | Obsidian |
3232
| **Meetings** | Zoom, Google Meet, Gong, Grain, Granola, Fathom, Fireflies |
3333
| **Recruiting** | Greenhouse, Ashby |
3434
| **Compliance** | Google Vault |
@@ -48,7 +48,6 @@ Other connectors use **API keys** or **personal access tokens** instead. The set
4848

4949
| Connector | Where to get the key |
5050
|-----------|---------------------|
51-
| **Evernote** | Developer Token (starts with `S=`) from your Evernote account settings |
5251
| **Obsidian** | Install the [Local REST API](https://github.com/coddingtonbear/obsidian-local-rest-api) plugin, then copy the key from its settings |
5352
| **Fireflies** | Generate from the Integrations page in your Fireflies account |
5453
| **Typeform** | Personal access token from your Typeform account settings |
@@ -57,7 +56,7 @@ Other connectors use **API keys** or **personal access tokens** instead. The set
5756
| **Amazon S3** | Secret Access Key (the Access Key ID, region, and bucket are entered as config fields) |
5857
| **Sentry** | Auth token with `project:read` and `event:read` scopes |
5958
| **PagerDuty** | REST API key from Integrations → API Access Keys |
60-
| **SFTP** | Password or unencrypted private key (host, port, username, and root path are entered as config fields) |
59+
| **SFTP** | Password or unencrypted private key, plus a required SHA-256 host key fingerprint (host, port, username, and root path are entered as config fields) |
6160
| **Mintlify** | API key — optional for public documentation sites, which sync from `llms.txt` |
6261

6362
<Callout type="info">

apps/sim/app/api/tools/docusign/route.ts

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -13,6 +13,7 @@ import {
1313
readResponseToBufferWithLimit,
1414
} from '@/lib/core/utils/stream-limits'
1515
import { withRouteHandler } from '@/lib/core/utils/with-route-handler'
16+
import { getDocusignOAuthUrl } from '@/lib/oauth/docusign'
1617
import { uploadCopilotFile } from '@/lib/uploads/contexts/copilot'
1718
import { uploadExecutionFile } from '@/lib/uploads/contexts/execution'
1819
import { FileInputSchema } from '@/lib/uploads/utils/file-schemas'
@@ -79,7 +80,7 @@ async function resolveAccount(
7980
signal?: AbortSignal
8081
): Promise<DocuSignAccountInfo> {
8182
const response = await fetchDocusign(
82-
'https://account-d.docusign.com/oauth/userinfo',
83+
getDocusignOAuthUrl('/oauth/userinfo'),
8384
{
8485
headers: { Authorization: `Bearer ${accessToken}` },
8586
},

0 commit comments

Comments
 (0)