fix: keep private and gated pages out of the search index - #6385
Merged
Conversation
Private squads still rendered as index,follow. getServerSideProps runs unauthenticated, so the API rejects every private squad with FORBIDDEN before the `noindex: !squad.public` line is reached, and the catch branch returned props with no seo at all, falling back to the indexable default. Same failure mode on the post page, where FORBIDDEN is what every post in a private squad returns from the unauthenticated ISR fetch. - squads/[handle]: noindex the error fallback, fail closed when `public` is not positively true (robots tags and JSON-LD alike) - posts/[id]: noindex the error fallback, fail closed for private posts and non-public sources, and drop the `robots` additionalMetaTags entry that escapes next-seo's dedupe (already covered by robotsProps) - noindex auth-gated squad surfaces: invite tokens, analytics, moderation - noindex the recruiter app, post analytics, org invite links, the auth flows (verification, reset-password, callback) and backoffice - regression tests for each path Claude-Session: https://claude.ai/code/session_01JP6oFXEXeuSsjr6VZQ6KMs
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
These pages were touched only to attach noindex seo; their strict errors predate this branch. Claude-Session: https://claude.ai/code/session_01JP6oFXEXeuSsjr6VZQ6KMs
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
Private squad pages were still served as
index, follow. #6322 fixed the robots-tag plumbing, but not the path private squads actually take.Root cause
getServerSidePropsruns unauthenticated, soensureSourcePermissionsin the API rejects every private squad withFORBIDDENon the firstsourcequery. That throws straight past thenoindex: !squad.publicline into thecatch, which returned{ props: { handle } }with noseo._apponly renders the page-levelNextSeowhen a page suppliesseo, so those pages fell back toDefaultSeoand advertised as indexable.The post page has the identical bug:
FORBIDDENis exactly what a post inside a private squad returns from the unauthenticated ISR fetch, and its catch branch also shipped noseo.Changes
The reported bug
squads/[handle]:noindex,nofollowon the error fallback (FORBIDDEN,NOT_FOUND, rate limited). Visibility check is nowsquad?.public === true, so an unknown/missingpublicfails closed for both the robots tags and the JSON-LD block.posts/[id]: same fallback fix;shouldNoindexPostalso fails closed forpost.privateand non-public sources, so a page cached before a squad went private cannot stay indexable.Same class, found while auditing the rest of the pages
posts/[id]/index.tsxreintroducedrobotsthroughadditionalMetaTags, which is what the comment innext-seo.tsexplicitly forbids: those are keyed by meta name, escape next/head's dedupe, and would silently override a page-levelnoindex. Removed. It was redundant withrobotsProps.maxImagePreview.seoat all:[handle]/analytics,squads/moderate, and[handle]/[token](personal invite links, private squads included).posts/[id]/analytics: author-only page that emitted a fully indexable seo block.recruiterSeois shared by all three recruiter layouts, so every dashboard, opportunity, candidate review, org settings and payment page saidindex,followwhile rendering nothing for a crawler.join/organization: token-bearing org invite links.verification,reset-password,callback) andbackoffice/*.Public squads, posts, feeds, sources, tags and profiles are untouched and keep
index,followwith themax-*directives.Verification
jsonLdleaks either), private/non-public posts, the post error fallback, and layout-level locks for the gated pages.tsc --noEmitreports nothing on the changed files. The strict-changed guard flags only pre-existing nullability backlog in the touched files, left alone.Not changed, flagging as judgment calls rather than deciding unilaterally:
embed/*,image-generator/*andpopup/*are non-content utility surfaces that are also indexable today.https://claude.ai/code/session_01JP6oFXEXeuSsjr6VZQ6KMs
Preview domain
https://fix-noindex-private-and-gated-pa.preview.app.daily.dev