feat: adding project-discovery-worker (CM-949)

[ulemons]

Summary

This PR introduces the basic structure for the new worker.

The implementation includes the foundational setup required to run the worker, without full business logic.

Included

• Worker initialization and configuration
• Base execution flow
• Placeholder structure for future processing logic

⚠️ This is just the basic structure of the worker, i won't probably merge this before the other prs of the POC will be completed

---

Note

Medium Risk
Introduces new database tables plus a scheduled Temporal worker that ingests large external datasets into Postgres, which can impact migrations, load, and operational stability if misconfigured. Most changes are additive but touch persistence, background scheduling, and deployment artifacts.

Overview
Adds automatic project discovery persistence and ingestion. This introduces new Postgres tables projectCatalog and evaluatedProjects (with indexes and down-migration), plus a data-access-layer for querying/inserting/upserting both entities.

Adds a new Temporal worker app automatic_projects_discovery_worker that schedules a daily discoverProjects workflow and processes external datasets (CSV from OSSF bucket and paginated JSON from an LF Criticality Score API) in batches, bulk-upserting into projectCatalog. Includes new Dockerfile/compose and build env for running/deploying the worker, along with lockfile updates for the new workspace package and dependencies.

^{Written by Cursor Bugbot for commit 75cb4b6. This will update automatically on new commits. Configure here.}

[Copilot]

Pull request overview

Introduces a new Temporal worker app (automatic_projects_discovery_worker) and adds initial database schema to support automatic project discovery/evaluation.

Changes:

• Added a new worker service with Temporal workflow/activity scaffolding plus a schedule that registers a cron-based Temporal Schedule.
• Added Docker and compose definitions to run the worker in the existing services stack.
• Added DB migrations for projectCatalog and evaluatedProjects tables.

Reviewed changes

Copilot reviewed 13 out of 14 changed files in this pull request and generated 2 comments.

Show a summary per file

File	Description
services/apps/automatic_projects_discovery_worker/tsconfig.json	Adds TS config for the new worker package.
services/apps/automatic_projects_discovery_worker/src/workflows/discoverProjects.ts	Introduces a placeholder Temporal workflow that calls a logging activity.
services/apps/automatic_projects_discovery_worker/src/workflows.ts	Exports the worker’s workflows for registration.
services/apps/automatic_projects_discovery_worker/src/schedules/scheduleProjectsDiscovery.ts	Registers a Temporal Schedule to trigger the workflow on a cron.
services/apps/automatic_projects_discovery_worker/src/main.ts	Worker bootstrap: init, schedule registration, start.
services/apps/automatic_projects_discovery_worker/src/activities/activities.ts	Placeholder activity implementation (logs a run).
services/apps/automatic_projects_discovery_worker/src/activities.ts	Barrel export for activities registration.
services/apps/automatic_projects_discovery_worker/package.json	Adds the new worker package and scripts.
scripts/services/docker/Dockerfile.automatic_projects_discovery_worker	Adds container build/run definition for the worker.
scripts/services/automatic-projects-discovery-worker.yaml	Adds docker-compose definitions for running the worker (prod + dev).
scripts/builders/automatic-projects-discovery-worker.env	Adds builder configuration for publishing/building the worker image.
pnpm-lock.yaml	Adds lockfile entries for the new workspace package (but currently includes an extra incorrect importer).
backend/src/database/migrations/V1770653666__add-automatic_projects_discovery-tables.sql	Adds tables/indexes for project discovery catalog + evaluations.
backend/src/database/migrations/U1770653666__add-automatic_projects_discovery-tables.sql	Adds down migration to drop the tables/indexes.

Files not reviewed (1)

• pnpm-lock.yaml: Language not supported

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[Copilot]

pnpm-lock.yaml contains two importers for this new app: services/apps/automatic_projects_discovery_worker (correct) and services/apps/automatic_project_discovery_worker (missing the s in projects). The latter path doesn’t exist in the repo and will leave the lockfile in an inconsistent state (pnpm will warn/error depending on settings). Please regenerate or edit the lockfile so only the correct importer remains.

[cursor]

Stale workspace importer in lockfile

Medium Severity

pnpm-lock.yaml adds an importer for services/apps/automatic_project_discovery_worker, but that workspace path is not present while the real app is services/apps/automatic_projects_discovery_worker. This leaves a stale duplicate importer in the lockfile, creating an inconsistent workspace definition that can destabilize pnpm install --frozen-lockfile behavior and dependency maintenance.

 

[cursor]

throw new Error(err) loses original error context

Low Severity

throw new Error(err) wraps the caught err (likely already an Error) inside a new Error, which discards the original stack trace, error type, and any custom properties. The resulting error message will be the stringified version of the original error (e.g., "Error: ...") rather than the original message. Rethrowing err directly would preserve all original context, making debugging much easier.

 

[CLAassistant]

Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.
_{You have signed the CLA already but the status is still pending? Let us recheck it.}

[cursor]

Cursor Bugbot has reviewed your changes and found 1 potential issue.

There are 3 total unresolved issues (including 2 from previous reviews).

^{Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, have a team admin enable autofix in the Cursor dashboard.}