fix(instagram): paginate following endpoint to handle limit > ~200 (#1831)#1835
Open
LeoLin990405 wants to merge 2 commits into
Open
fix(instagram): paginate following endpoint to handle limit > ~200 (#1831)#1835LeoLin990405 wants to merge 2 commits into
LeoLin990405 wants to merge 2 commits into
Conversation
Closes jackwener#1831. The `following` adapter was issuing a single fetch to `/api/v1/friendships/{userId}/following/?count={limit}` and passing the user-supplied `limit` straight through. Instagram's friendships endpoint returns HTTP 400 for `count` values above ~200, so any account followed by more than 200 users could not be fully listed. Replace the single fetch with a pagination loop: - Fixed page size `count=50` (safely under the API ceiling) - Follow the `next_max_id` cursor - Dedupe by `pk` to defend against cursor overlap - ~400 ms pacing between pages - Stop on `limit` reached, empty page, or missing cursor Verified against an account following 296 — full list now returns 295 unique entries (vs. HTTP 400 before). Tests - New `clis/instagram/instagram.test.js`, 5 cases: - single page < 50 - 3 pages via next_max_id (rank continuity) - dedupe by pk on cursor overlap - limit respected (stops early) - empty following returns [] - `npm run test:adapter` — 380 files / 3719 tests pass
…ro-growth pages Address review feedback (coder review NEEDS FIX): - Add `seenCursors` Set so a repeating `next_max_id` aborts the loop. - Bail out when a full page yields zero new unique users (e.g. the API echoes the previous page after dedupe). - Skip the final 400 ms sleep when `limit` is already reached. Tests - `limit=0` short-circuits before the following endpoint. - Repeating `next_max_id` aborts after the second observation. - Zero-growth page (page-2 dedupes to empty) aborts. - HTTP error on a mid-pagination page propagates as before. - `globalThis.fetch` saved/restored instead of deleted. `npx vitest run clis/instagram/instagram.test.js` — 9/9 pass.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #1831.
What
opencli instagram following <user> --limit Nissued a single fetch with the user-suppliedlimitpassed straight through ascountto/api/v1/friendships/{userId}/following/. Instagram returns HTTP 400 forcountvalues above ~200, so any account followed by more than ~200 users could not be fully listed.Fix
Replace the single fetch with a pagination loop:
count=50(safely under the API ceiling)next_max_idcursorpk(defends against cursor overlap)limitreached, missing cursor, empty page, repeated cursor, or a page that yields zero new unique users (avoids any infinite-loop edge if the API echoes itself)Verification
npx vitest run clis/instagram/instagram.test.js— 9/9 passnpm run test:adapter— 380 files / 3723 tests pass, no regressionnpm run build-manifest— idempotent (no CLI-surface change)Tests added (
clis/instagram/instagram.test.js, new file)next_max_id(rank continuity across 50+50+20)pkon cursor overlaplimitrespected — stops early, no extra fetch[]next_max_id→ cursor-loop guard breaks (3 calls instead of 4)limit=0short-circuits before the following endpointFailed to fetch following: HTTP 429Behavior
--limitCLI surface unchanged — this is a behind-the-scenes fix, no flag added, no README/manifest delta.