Skip to content

feat: add OSS OpenSearch and semantic FTS support#815

Open
jamesgao-jpg wants to merge 25 commits into
zilliztech:mainfrom
jamesgao-jpg:fts_v2_backend
Open

feat: add OSS OpenSearch and semantic FTS support#815
jamesgao-jpg wants to merge 25 commits into
zilliztech:mainfrom
jamesgao-jpg:fts_v2_backend

Conversation

@jamesgao-jpg

@jamesgao-jpg jamesgao-jpg commented Jul 14, 2026

Copy link
Copy Markdown
Collaborator

Summary

This PR adds OSS OpenSearch and the backend support needed for both unfiltered and filtered full-text search benchmarking. Filtered FTS frontend presentation and generated result publication remain deferred.

PR #794 introduced the preceding full-text search benchmark foundation used by this work. This PR supersedes #814 with a reduced publication scope. PR #818 introduces shared CLI run-context notes; its commit is incorporated here so these FTS runs can preserve deployment and constraint context in result JSONs.

Changes

  • add the OSS OpenSearch backend client, configuration, CLI registration, and focused tests
  • retain unfiltered semantic FTS metrics and controls
  • restore filtered FTS backend cases, dataset preparation, runner orchestration, metrics, and CLI --fts-filter-rate control
  • distribute FTS filter_id values with a deterministic bijection so exact-selectivity filters are scattered across corpus insertion order rather than selecting a contiguous suffix
  • expose --fts-filter-id-distribution [permuted|sequential], defaulting to permuted, and propagate the selected mode through case construction, dataset preparation, qrel filtering, and result metadata
  • restore filtered FTS request translation for Milvus, Elastic Cloud, OSS OpenSearch, TurboPuffer, and Vespa
  • keep the standard FTS QPS, recall, NDCG, MRR, and load frontend views without exposing filtered FTS cases
  • report OSS OpenSearch bulk item failures with the actual successful-document count instead of treating partial writes as full success
  • add shared --note and --note-file CLI options while preserving Pinecone compatibility
  • preserve current upstream backend and compatibility changes by incorporating the latest main

Deferred Scope

  • filtered FTS frontend controls and presentation
  • generated filtered FTS result JSON artifacts
  • session and documentation-only migration files

The filtered FTS backend and CLI paths are included, but filtered results are not published in the frontend by this PR.

Compatibility

FTS collections loaded before commit bf85cf7 contain sequential filter_id values. Reload those collections before running filtered FTS search so stored IDs and semantic qrel filtering use the same mapping. Changing --fts-filter-id-distribution between permuted and sequential also requires reloading the collection.

Verification

  • make lint
  • make unittest
  • 75 focused filtered-FTS backend, case, CLI, dataset, runner, OpenSearch, and TurboPuffer tests
  • DCO verification for every outgoing commit
  • VDBBench pre-push gate and impact-map validation
  • exact production restoration recorded in commit 659f739, followed by isolated current-lint compatibility commit a352b73
  • deterministic exact-selectivity FTS filter distribution added in commit bf85cf7
  • explicit sequential/permuted CLI and case switch added in commit e24110b

No live provider probe was run for this update; provider request translation is covered by mocked unit tests.

Signed-off-by: jamesgao-jpg <james.gao@zilliz.com>
Signed-off-by: jamesgao-jpg <james.gao@zilliz.com>
Signed-off-by: jamesgao-jpg <james.gao@zilliz.com>
Signed-off-by: jamesgao-jpg <james.gao@zilliz.com>
Signed-off-by: jamesgao-jpg <james.gao@zilliz.com>
Signed-off-by: jamesgao-jpg <james.gao@zilliz.com>
Signed-off-by: jamesgao-jpg <james.gao@zilliz.com>
Signed-off-by: jamesgao-jpg <james.gao@zilliz.com>
Signed-off-by: jamesgao-jpg <james.gao@zilliz.com>
Signed-off-by: jamesgao-jpg <james.gao@zilliz.com>
Signed-off-by: jamesgao-jpg <james.gao@zilliz.com>
Signed-off-by: jamesgao-jpg <james.gao@zilliz.com>
Signed-off-by: jamesgao-jpg <james.gao@zilliz.com>
Signed-off-by: jamesgao-jpg <james.gao@zilliz.com>
Signed-off-by: jamesgao-jpg <james.gao@zilliz.com>
Signed-off-by: jamesgao-jpg <james.gao@zilliz.com>
Remove filtered FTS dataset, runner, backend, frontend, test, and published
result changes while retaining unfiltered OSS OpenSearch and semantic FTS
support. Remove the task-local session progress file from the branch.

Signed-off-by: jamesgao-jpg <james.gao@zilliz.com>
Preserve current upstream backends and compatibility changes while retaining
the unfiltered FTS and OSS OpenSearch work from fts_ver2.

Signed-off-by: jamesgao-jpg <james.gao@zilliz.com>
@sre-ci-robot

Copy link
Copy Markdown

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by: jamesgao-jpg
To complete the pull request process, please assign xuanyang-cn after the PR has been reviewed.
You can assign the PR to them by writing /assign @xuanyang-cn in a comment when ready.

The full list of commands accepted by this bot can be found here.

Details Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

jamesgao-jpg

This comment was marked as outdated.

Signed-off-by: jamesgao-jpg <james.gao@zilliz.com>
Signed-off-by: jamesgao-jpg <james.gao@zilliz.com>
Signed-off-by: jamesgao-jpg <james.gao@zilliz.com>
Restore the filtered full-text search cases, datasets, runners, backend
clients, CLI controls, and regression tests removed by 7cd3046.

Keep frontend support, generated result artifacts, session files, and
documentation excluded from this backend migration.

Signed-off-by: jamesgao-jpg <james.gao@zilliz.com>
Apply current Black formatting to the restored filtered-FTS code, use direct
integer filter attribute access, and document the intentional runtime error
inside the serial-search exception boundary.

Signed-off-by: jamesgao-jpg <james.gao@zilliz.com>
Assign FTS filter IDs through a versioned affine permutation so threshold
filters preserve exact cardinality while spreading matches across corpus
insertion order.

Use the same mapping for semantic qrels and record its parameters in filter
statistics for reproducibility.

Signed-off-by: jamesgao-jpg <james.gao@zilliz.com>
Expose sequential and permuted FTS filter-ID distributions through the common
CLI and FTS case configuration, defaulting to the deterministic permutation.

Propagate the selected mode into document insertion, semantic qrel filtering,
and versioned result metadata. Changing modes requires reloading the target
collection.

Signed-off-by: jamesgao-jpg <james.gao@zilliz.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants