Develop#229
Merged
Merged
Conversation
…ct and datasets for landing pages
…l missing projects) in order to retrieve downloads with their appr sequences and other possible routes
…l missing projects) in order to retrieve downloads with their appr sequences and other possible routes
…st local fixes to allow me to connect to DB locally. Merge remote-tracking branch 'origin/production' into nick_datasetquery insert �
Nick datasetquery
queries, download sequences, and
remove /whoami
- datasetqueryfaster.sql: Remove early
LIMIT/OFFSET hack in
filtered_datasets
CTE. The 3x multiplier produced
unstable pagination; final
LIMIT/OFFSET
on the main query handles pagination
correctly.
- pubdsidquery.sql: Restore CTE
pre-filter on datasetpublications for
performance. Change INNER JOIN to
LEFT JOIN on publicationauthors,
contacts, and publicationtypes so
publications missing that data are
no longer silently dropped.
- publications.js: Remove dead
aggregation code in
publicationbydataset
handler (returner was built but
never sent; response already used
raw bibOutput).
lDataIds
before querying sequencedata to before queruce before querying sequencedata to bloads before querying sequencedata to befo before querying sequencedata to uenc before querying sequencedata to beforn only the highest
(most recent) (most recent) (most recent) enceid column.
- Remove /v2.0/apps/whoami endpoint;
replaced by /v2.0/apps/orcids/me.
Aedna/endpoints
There was a problem hiding this comment.
Pull request overview
This PR expands the v2.0 API by adding new data endpoints for (1) publications authored by a contact and (2) aeDNA sequence retrieval (by dataset and by taxon), and updates the OpenAPI spec and test suite accordingly. It also adds sequence enrichment to the download response and extends contact/auth responses with steward/session metadata.
Changes:
- Add
/v2.0/data/contacts/{contactid}/publicationsand/v2.0/data/aedna/...endpoints (routes, handlers, SQL helpers) plus OpenAPI + tests. - Enrich download payloads with sequence strings for datum entries.
- Add
stewardidto contact lookups and auth “me” response; adjust CORS/dev origin handling and CI deploy action versions.
Reviewed changes
Copilot reviewed 55 out of 58 changed files in this pull request and generated 3 comments.
Show a summary per file
| File | Description |
|---|---|
| v2.0/routes/data.js | Registers new contact-publications and aeDNA routes. |
| v2.0/routes/apps.js | Removes unused /whoami; continues to expose /orcids/me for session identity. |
| v2.0/helpers/publications/publications.js | Simplifies dataset-publications response payload handling. |
| v2.0/helpers/publications/pubdsidquery.sql | Refactors joins for dataset-publication JSON query. |
| v2.0/helpers/download/sequencebydata.sql | Adds query to fetch latest sequence string per dataid. |
| v2.0/helpers/download/download.js | Adds sequence enrichment to download response payload. |
| v2.0/helpers/datasets/datasetqueryfaster.sql | Changes dataset query pagination behavior inside CTE. |
| v2.0/helpers/contacts/pubbycontactid.sql | Adds SQL to fetch publications authored by a contact. |
| v2.0/helpers/contacts/contacts.js | Adds handler for contact publications endpoint. |
| v2.0/helpers/contacts/contactquery.sql | Adds stewardid to contact query results. |
| v2.0/helpers/contacts/contactbyid.sql | Adds stewardid to contact-by-id results. |
| v2.0/helpers/aedna/sequencesbytaxon.sql | Adds aeDNA sequences query by taxon. |
| v2.0/helpers/aedna/sequences.sql | Adds aeDNA sequences query by dataset (grouped by taxon + taxon chain). |
| v2.0/helpers/aedna/aedna.js | Adds Express handlers for aeDNA endpoints. |
| v2.0/handlers/data_handlers.js | Wires new data handlers to helper modules. |
| v2.0/handlers/apps_handlers.js | Extends /orcids/me response with expiresat + stewardid. |
| test/v2.0-data-taxa-test.js | Updates generated test query parameters. |
| test/v2.0-data-speleothems-{collectionunitid}-test.js | Updates generated test ID. |
| test/v2.0-data-spatial-lakes-test.js | Updates generated test query parameters. |
| test/v2.0-data-spatial-icesheet-test.js | Updates generated test query parameters. |
| test/v2.0-data-spatial-faunal-test.js | Updates generated test query parameters. |
| test/v2.0-data-sites-test.js | Updates generated test query parameters. |
| test/v2.0-data-sites-{siteid}-test.js | Updates generated test ID. |
| test/v2.0-data-sites-{siteid}-contacts-test.js | Updates generated test ID. |
| test/v2.0-data-publications-test.js | Updates generated test query parameters. |
| test/v2.0-data-publications-{publicationid}-test.js | Updates generated test ID. |
| test/v2.0-data-pollen-test.js | Updates generated test query parameters. |
| test/v2.0-data-pollen-{id}-test.js | Updates generated test ID. |
| test/v2.0-data-occurrences-test.js | Updates generated test query parameters. |
| test/v2.0-data-geopoliticalunits-test.js | Updates generated test query parameters. |
| test/v2.0-data-geopoliticalunits-{gpid}-test.js | Updates generated test ID. |
| test/v2.0-data-geopoliticalunits-{gpid}-sites-test.js | Updates generated test ID. |
| test/v2.0-data-geopoliticalunits-{gpid}-datasets-test.js | Updates generated test ID. |
| test/v2.0-data-dbtables-test.js | Updates generated test query parameters. |
| test/v2.0-data-dbtables-{table}-test.js | Updates generated test table name. |
| test/v2.0-data-datasets-test.js | Updates generated test query parameters. |
| test/v2.0-data-datasets-db-test.js | Updates generated test query parameters. |
| test/v2.0-data-datasets_elc-test.js | Updates generated test query parameters. |
| test/v2.0-data-contacts-test.js | Updates generated test query parameters. |
| test/v2.0-data-contacts-{contactid}-test.js | Updates generated test ID. |
| test/v2.0-data-contacts-{contactid}-sites-test.js | Updates generated test ID. |
| test/v2.0-data-contacts-{contactid}-publications-test.js | Adds tests for new contact publications endpoint. |
| test/v2.0-data-aggregatedatasets-{aggdatasetid}-test.js | Updates generated test ID. |
| test/v2.0-data-aedna-taxa-{taxonid}-sequences-test.js | Adds tests for new aeDNA-by-taxon endpoint. |
| test/v2.0-data-aedna-sequences-{datasetid}-test.js | Adds tests for new aeDNA-by-dataset endpoint. |
| test/v2.0-apps-depenvt-test.js | Updates generated test query parameters. |
| test/v2.0-apps-constdb-datasetuploads-test.js | Updates generated test query parameters. |
| test/v2.0-apps-constdb-datasets-test.js | Updates generated test query parameters. |
| test/v1.5-data-geopoliticalunits-test.js | Updates generated test query parameters. |
| test/v1.5-data-geopoliticalunits-{gpid}-test.js | Updates generated test ID. |
| test/v1.5-data-contacts-{contactid}-test.js | Updates generated test ID. |
| openapi/paths/v20/data.yml | Documents new contact-publications and aeDNA endpoints. |
| openapi.yaml | Adds the same OpenAPI paths to the bundled spec. |
| app.js | Tightens CORS allow-list and allows localhost origins in non-production via regex. |
| .gitignore | Ignores tasks/ directory. |
| .github/workflows/deploy.yml | Bumps checkout + AWS credentials actions major versions. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
Comment on lines
+38
to
+41
| INNER JOIN dpub ON dpub.publicationid = pub.publicationid | ||
| LEFT JOIN ndb.publicationauthors AS pa ON pub.publicationid = pa.publicationid | ||
| LEFT JOIN ndb.contacts as ca ON ca.contactid = pa.contactid | ||
| LEFT JOIN ndb.publicationtypes AS pt ON pub.pubtypeid = pt.pubtypeid |
Comment on lines
26
to
+29
| -- This is cheating a bit. We're querying a smaller chunk here and hoping that taking 3 times the datasets is enough | ||
| -- to get us the actual limit. | ||
| LIMIT COALESCE(${limit}, 25) * 3 | ||
| OFFSET COALESCE(${offset}, 0) | ||
| -- LIMIT COALESCE(${limit}, 25) * 3 | ||
| -- OFFSET COALESCE(${offset}, 0) |
| res.status(200) | ||
| .json({ | ||
|
|
||
| // Collect all dataids from datum objects to look up sequenceids |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
No description provided.