docs(web-analytics): add guide for managing bot and AI traffic

[lricoy]

Changes

Fills the task-oriented gaps around bot/AI traffic in the web analytics docs. The existing bot detection page is a solid function reference; this adds the "what should I do about bots?" guidance and brings two adjacent pages up to date.

• New page – Managing bot and AI traffic (managing-bot-traffic.mdx): decision-oriented guide covering which traffic to keep vs exclude, the three places to act (client-side block / query-time exclude / server-side $http_log capture), how to exclude bots from an insight or dashboard, and how to measure AI crawler traffic. Links out to the reference rather than repeating functions. Added to the web analytics nav.
• Live (live.mdx): documents the bot traffic surfaces — the bot requests per minute chart and the bot traffic tile (ranked, categorized, drill-into-a-bot). Notes that the live bot tiles are still rolling out (behind WEB_ANALYTICS_BOT_ANALYSIS), while query-time classification is GA today.
• Troubleshooting (troubleshooting.mdx): the "Do stats include bots?" answer no longer implies client-side blocking is the whole story — it now explains that most crawlers/AI agents never run JS, and points to server-side $http_log classification. Linked the dedicated bot detection page from the HogQL subsection.

Checklist

• I've read the docs and/or content style guides.
• Words are spelled using American English
• Use relative URLs for internal links
• I've checked the pages added or changed in the Vercel preview build
• No pages moved (new page, no redirect needed)

Reviewer note: the Live "Bot traffic" section describes flag-gated UI. Time the merge with the WEB_ANALYTICS_BOT_ANALYSIS rollout, or I can gate that section's wording further if you'd rather ship the GA bits first.

[github-actions]

Deploy preview

Status	Details	Updated (UTC)
🟢 Ready	View preview	Jun 22, 2026 11:58PM

[github-actions]

Vale prose linter → found 3 errors, 31 warnings, 10 suggestions in your markdown

Full report → Copy the linter results into an LLM to batch-fix issues.

Linter being weird? Update the rules!

contents/docs/web-analytics/live.mdx — 0 errors, 1 warnings, 0 suggestions

Line	Severity	Message	Rule
47:4	warning	'Live vs. the rest of Web Analytics' heading should be in sentence case, and product names should be capitalized.	PostHogBase.SentenceCase

contents/docs/web-analytics/managing-bot-traffic.mdx — 0 errors, 5 warnings, 1 suggestions

Line	Severity	Message	Rule
13:19	warning	'GPTBot' is a possible misspelling.	PostHogBase.Spelling
14:25	warning	'Googlebot' is a possible misspelling.	PostHogBase.Spelling
14:36	warning	'Bingbot' is a possible misspelling.	PostHogBase.Spelling
15:57	warning	'Pingdom' is a possible misspelling.	PostHogBase.Spelling
26:164	warning	Capitalize 'Logs' for PostHog's product. Use 'logs' for the general industry concept.	PostHogBase.ProductNames
26:285	suggestion	Address the reader directly. Use 'you' instead of 'the user'.	PostHogDocs.DirectAddress

contents/docs/web-analytics/troubleshooting.mdx — 3 errors, 25 warnings, 9 suggestions

Line	Severity	Message	Rule
12:4	warning	'Have a question? Ask PostHog AI' heading should be in sentence case, and product names should be capitalized.	PostHogBase.SentenceCase
16:60	warning	Use 'PostHog' instead of 'posthog'.	Vale.Terms
18:4	warning	'What is the difference between web analytics and product analytics?' heading should be in sentence case, and product names should be capitalized.	PostHogBase.SentenceCase
18:53	warning	Capitalize 'Product Analytics' for PostHog's product. Use 'product analytics' for the general industry concept.	PostHogBase.ProductNames
20:108	warning	Capitalize 'Product Analytics' for PostHog's product. Use 'Product analytics' for the general industry concept.	PostHogBase.ProductNames
22:82	warning	Capitalize 'Product Analytics' for PostHog's product. Use 'product analytics' for the general industry concept.	PostHogBase.ProductNames
24:4	warning	'How much does web analytics cost?' heading should be in sentence case, and product names should be capitalized.	PostHogBase.SentenceCase
26:47	warning	Capitalize 'Product Analytics' for PostHog's product. Use 'product analytics' for the general industry concept.	PostHogBase.ProductNames
26:108	warning	Capitalize 'Product Analytics' for PostHog's product. Use 'product analytics' for the general industry concept.	PostHogBase.ProductNames
30:226	warning	Capitalize 'Data Pipelines' for PostHog's product. Use 'Data pipelines' for the general industry concept.	PostHogBase.ProductNames
36:243	warning	Capitalize 'Logs' for PostHog's product. Use 'logs' for the general industry concept.	PostHogBase.ProductNames
38:5	warning	'Analyzing bot traffic with HogQL' heading should be in sentence case, and product names should be capitalized.	PostHogBase.SentenceCase
44:64	suggestion	Address the reader directly. Use 'you' instead of 'the user'.	PostHogDocs.DirectAddress
47:78	warning	'Googlebot' is a possible misspelling.	PostHogBase.Spelling
82:1	warning	'Cookieless' is a possible misspelling.	PostHogBase.Spelling
82:148	suggestion	Address the reader directly. Use 'you' instead of 'the user'.	PostHogDocs.DirectAddress
82:334	suggestion	Address the reader directly. Use 'you' instead of 'the user'.	PostHogDocs.DirectAddress
84:7	warning	'cookieless' is a possible misspelling.	PostHogBase.Spelling
84:111	suggestion	Address the reader directly. Use 'you' instead of 'the user'.	PostHogDocs.DirectAddress
90:176	suggestion	Address the reader directly. Use 'you' instead of 'the user'.	PostHogDocs.DirectAddress
96:71	warning	'archivers' is a possible misspelling.	PostHogBase.Spelling
100:185	error	Hi, Andy here... use an en dash ( – ) with spaces. On Mac, holding down the Option and hyphen key will give you an en dash.	PostHogBase.EnDash
102:24	warning	'Ahrefs' is a possible misspelling.	PostHogBase.Spelling
102:100	suggestion	Address the reader directly. Use 'you' instead of 'the user'.	PostHogDocs.DirectAddress
122:339	suggestion	Address the reader directly. Use 'you' instead of 'the user'.	PostHogDocs.DirectAddress
134:4	suggestion	Address the reader directly. Use 'you' instead of 'The user'.	PostHogDocs.DirectAddress
136:4	suggestion	Address the reader directly. Use 'you' instead of 'The user'.	PostHogDocs.DirectAddress
142:4	warning	'Why does PostHog impact my PageSpeed score?' heading should be in sentence case, and product names should be capitalized.	PostHogBase.SentenceCase
148:127	warning	Capitalize 'Session Replay' for PostHog's product. Use 'session replay' for the general industry concept.	PostHogBase.ProductNames
148:145	warning	Capitalize 'Surveys' for PostHog's product. Use 'surveys' for the general industry concept.	PostHogBase.ProductNames
154:5	warning	'Why you shouldn't care about PageSpeed score' heading should be in sentence case, and product names should be capitalized.	PostHogBase.SentenceCase
158:194	warning	'Mbps' is a possible misspelling.	PostHogBase.Spelling
158:252	warning	'Mbps' is a possible misspelling.	PostHogBase.Spelling
166:26	error	Use straight quotes and apostrophes, not curly ones.	PostHogDocs.CurlyQuotes
166:34	error	Use straight quotes and apostrophes, not curly ones.	PostHogDocs.CurlyQuotes
170:5	warning	'Sorry, you didn't convince me. I still care about PageSpeed score. What do I do?' heading should be in sentence case, and product names should be capitalized.	PostHogBase.SentenceCase
198:296	warning	'precompute' is a possible misspelling.	PostHogBase.Spelling

[github-actions]

Bundle report

Total JS (gzip)

6.21 MiB (+0.0 KiB / +0.0%)

Eager graph (static-import closure per entrypoint)

Entrypoint	Eager size	Budget	Modules
✅ app	24.11 MiB (+0.6 KiB / +0.0%)	report-only	5502

Largest modules in the app closure

Module	Size
css ./node_modules/.pnpm/css-loader@5.2.7_webpack@5.101.3/node_modules/css-loader/dist/cjs.js??ruleSet[1].rules[8].oneOf[1].use[1]!./node_modules/.pnpm/postcss-loader@4.3.0_postcss@8.5.6_webpack@5.101.3/node_modules/postcss-loader/dist/cjs.js??ruleSet[1].rules[8].oneOf[1].use[2]!./src/styles/global.css	709.9 KiB
./src/components/Stickers/Stickers.tsx	696.4 KiB
./.cache/caches/gatsby-plugin-mdx/mdx-scopes-dir/31a094f140f119e73085d847ae81b99b.js + 2 modules	519.2 KiB
./node_modules/.pnpm/@radix-ui+react-icons@1.3.2_react@18.3.1/node_modules/@radix-ui/react-icons/dist/react-icons.esm.js	481.4 KiB
./node_modules/.pnpm/@codemirror+view@6.38.2/node_modules/@codemirror/view/dist/index.js	458.1 KiB
./node_modules/.pnpm/rehype-raw@7.0.0/node_modules/rehype-raw/lib/index.js + 29 modules	395.1 KiB
./node_modules/.pnpm/@posthog+icons@0.36.6_react-dom@18.3.1_react@18.3.1__react@18.3.1/node_modules/@posthog/icons/dist/posthog-icons.cjs.js	364.8 KiB
./node_modules/.pnpm/@posthog+icons@0.36.6_react-dom@18.3.1_react@18.3.1__react@18.3.1/node_modules/@posthog/icons/dist/posthog-icons.es.js	354.8 KiB
./src/hooks/useCustomers.tsx + 54 modules	353.9 KiB
./node_modules/.pnpm/react-markdown@8.0.7_@types+react@16.14.66_react@18.3.1/node_modules/react-markdown/lib/react-markdown.js + 88 modules	351.4 KiB
./node_modules/.pnpm/cloudinary-core@2.14.0_lodash@4.17.21/node_modules/cloudinary-core/cloudinary-core.js	281.9 KiB
./node_modules/.pnpm/@codesandbox+sandpack-react@2.20.0_react-dom@18.3.1_react@18.3.1__react@18.3.1/node_modules/@codesandbox/sandpack-react/dist/index.mjs	266.6 KiB
./src/components/ProductComparisonTable/index.tsx + 114 modules	264.0 KiB
./node_modules/.pnpm/d3@7.9.0/node_modules/d3/src/index.js + 208 modules	247.4 KiB
./src/components/Pricing/PricingSlider/Slider.tsx + 87 modules	239.9 KiB

---

Eager-graph budgets are report-only until a baseline is established. Sizes are gzip of public/**/*.js; eager size is webpack module source bytes.

[lricoy]

@inkeep review this please

[inkeep]

Hey @lricoy 👋 I've reviewed your documentation PR and it looks great overall! The content is well-structured, follows our style guide, and does a nice job of providing decision-oriented guidance while linking out to the existing reference page.

What looks good ✅

• Clear task-oriented structure – The "Managing bot and AI traffic" page effectively answers "what should I do about bots?" rather than just listing functions
• Smart content separation – Good call linking to the bot detection reference page rather than duplicating function details
• Helpful decision table – The "Decide what to keep" table gives actionable guidance for different traffic types
• Proper cross-linking – Related pages are well-connected throughout

Minor suggestions

1. Frontmatter for the new page

The managing-bot-traffic.mdx page is missing standard frontmatter fields. Consider adding:

---
title: Managing bot and AI traffic
sidebar: Docs
showTitle: true
availability:
  free: full
  selfServe: full
  enterprise: full
---

2. Flag-gated content note

You've noted in the PR description that the Live "Bot traffic" section describes flag-gated UI (WEB_ANALYTICS_BOT_ANALYSIS). The current note handles this well:

Note: The live bot tiles are rolling out and may not be enabled for your project yet.

This is a reasonable approach for shipping the docs alongside the rollout. Just wanted to confirm you're comfortable with this timing.

---

Overall, this is ready to ship from a docs quality perspective. Nice work! 🚀