Skip to content

Nick datasetquery #226

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
hoffmanick wants to merge 3 commits into from
Closed

Nick datasetquery #226

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
eb125cd
update for stable pagination
hoffmanick May 20, 2026
365bd75
Update pubsidquery to show all pubs
hoffmanick May 29, 2026
7f6e1f9
Doing this because I need to get merge to work. app.js changes are ju…
hoffmanick Jun 2, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions v2.0/helpers/datasets/datasetqueryfaster.sql
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -25,8 +25,8 @@ WITH filtered_datasets AS (
ORDER BY bigq.siteid, bigq.datasetid
-- This is cheating a bit. We're querying a smaller chunk here and hoping that taking 3 times the datasets is enough
-- to get us the actual limit.
LIMIT COALESCE(${limit}, 25) * 3
OFFSET COALESCE(${offset}, 0)
-- LIMIT COALESCE(${limit}, 25) * 3
-- OFFSET COALESCE(${offset}, 0)
),
-- Now these CTEs work on a much smaller dataset
dataset_dois AS (
Expand Down
15 changes: 6 additions & 9 deletions v2.0/helpers/publications/pubdsidquery.sql
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,7 +1,3 @@
WITH dpub AS
(SELECT * FROM
ndb.datasetpublications as dp
WHERE ($1 IS NULL OR dp.datasetid IN ($1:csv)))
SELECT json_build_object(
'datasets', json_agg(DISTINCT jsonb_build_object('siteid', dsl.siteid,
'datasetid', dpub.datasetid,
Expand Down Expand Up @@ -35,9 +31,10 @@ SELECT json_build_object(
'givennames', ca.givennames,
'order', pa.authororder))) AS publication
FROM ndb.publications AS pub
INNER JOIN ndb.publicationauthors AS pa ON pub.publicationid = pa.publicationid
INNER JOIN ndb.contacts as ca ON ca.contactid = pa.contactid
INNER JOIN ndb.publicationtypes AS pt ON pub.pubtypeid = pt.pubtypeid
INNER JOIN (SELECT * FROM dpub) AS dpub ON dpub.publicationid = pub.publicationid
LEFT JOIN ndb.publicationauthors AS pa ON pub.publicationid = pa.publicationid
LEFT JOIN ndb.contacts as ca ON ca.contactid = pa.contactid
LEFT JOIN ndb.publicationtypes AS pt ON pub.pubtypeid = pt.pubtypeid
LEFT JOIN ndb.datasetpublications as dpub ON dpub.publicationid = pub.publicationid
LEFT JOIN ndb.dslinks AS dsl ON dsl.datasetid = dpub.datasetid
GROUP BY pub.publicationid, pt.pubtype
WHERE ($1 IS NULL OR dpub.datasetid IN ($1:csv))
GROUP BY pub.publicationid, pt.pubtype