diff --git a/content/en/documentation/for-contributors/testing.md b/content/en/documentation/for-contributors/testing.md index c46dded..16bc7c7 100644 --- a/content/en/documentation/for-contributors/testing.md +++ b/content/en/documentation/for-contributors/testing.md @@ -135,8 +135,7 @@ command: ### 4.5 Running them in the CI pipeline -All integration tests should go into the [`verify.yaml` workflow](/.github/workflows/verify.yaml), every "technology" -should +All integration tests should go into the `verify.yaml` workflow, every "technology" should have its own job, and technology specific tests can be targeted using Junit tags with `-DincludeTags` property as described above in document. diff --git a/developer/decision-records/2026-03-02-catalog-crawler/README.md b/developer/decision-records/2026-03-02-catalog-crawler/README.md new file mode 100644 index 0000000..c3ea792 --- /dev/null +++ b/developer/decision-records/2026-03-02-catalog-crawler/README.md @@ -0,0 +1,28 @@ +# Catalog Crawler as Control-Plane feature + +## Decision +The catalog crawler implementation currently contained in the [`FederatedCatalog`](https://github.com/eclipse-edc/FederatedCatalog) +will be moved into the [`Connector`](https://github.com/eclipse-edc/Connector) repository. + +## Rationale +It has been observed that the EDC `Federated Catalog` - the EDC component that provides the catalog crawling feature, +either as part of the control-plane or as a separate runtime - is lacking adoption. However, it is considered by the EDC +committer group to be one of the most crucial components in dataspace interactions. + +While, for example and testing purposes, remotely retrieving catalogs from counterparties to read through contract offers +may seem like a good pattern to follow, in the context of production dataspaces with thousands of participants, this +approach could result in slowdowns in the data retrieval process and cause network overloads. + +Crawling and caching catalogs locally can also enable a set of features based on more refined search queries that could +be executed repeatedly and quickly on the local dataset. + +We believe this feature is underused because the `Federated Catalog` exists as a separate repository, and there are +currently no examples in which the crawling feature is embedded within a control-plane. + +## Approach + +All modules in the `FederatedCatalog` repository (including BOMs) will be moved into the `Connector` repository. The +`FederatedCatalog` repository will then be archived. + +The catalog crawling feature will be enabled by default in the `controlplane-base-bom` module, with appropriate test +coverage and a configuration setting that will allow the crawling feature to be disabled if needed. \ No newline at end of file diff --git a/developer/decision-records/README.md b/developer/decision-records/README.md index 4d215d2..7c2ed96 100644 --- a/developer/decision-records/README.md +++ b/developer/decision-records/README.md @@ -30,7 +30,8 @@ - [2024-08-24 New Release process](./2024-08-26-new-release-process/) - [2024-09-19 Bills of Material](./2024-09-19-bills-of-material/) - [2024-10-11 Dependencies Check](./2024-10-11-dependencies-check/) -- [2026-02-25 On the usage of Artifical Intelligence](./2026-02-25-using-ai/) +- [2026-02-25 On the usage of Artificial Intelligence](./2026-02-25-using-ai/) +- [2026-03-22 Catalog Crawler as Control-Plane feature](./2026-03-02-catalog-crawler/) ## Repository-specific