[oncall-telemetry] Default report window to rolling 7 days ending today, Fixes AB#3683194

[shahzaibj]

Fixes AB#3683194

Feedback

From Fadi Durah's ★4 skill-effectiveness survey (Jun 2026, oncall-weekly-telemetry-report):

"The telemetry skill was certainly helpful, the biggest feedback is that it seemed to capture the wrong timeframe (didn't capture the most recent 6 days). I'm sure I could've spent some time with it to get it to target the right date, but this is probably something that should happen by default in order to save more time for the developer, and something moumita has been pushing for."

Root cause (repro'd on today's clock, Thu 2026-07-09)

bootstrap-report.ps1 was computing "reporting Sunday" as the most recent Sunday strictly before today — which is the Sunday of the currently in-progress week. Run on a Thursday it picked Sun 2026-07-05 → Sat 2026-07-11 where Saturday is 2 days in the future, so the WoW section queried a 4-day partial window and dropped the last complete week. Exactly Fadi's "missed the last 6 days" complaint. SKILL.md then compounded it by asking the model to confirm the Sunday with the user — the manual coax that the developer shouldn't have to do.

Today (UTC)	Old default (broken)	New default
Thu 2026-07-09	Sun 07-05 → Sat 07-11 (future!)	Thu 07-02 → Thu 07-09 ✅
Mon 2026-07-13	Sun 07-12 → Sat 07-18 (future!)	Mon 07-06 → Mon 07-13 ✅

Fix

Per the user's direction, deep rewrite: rolling [today−7d, today) for the primary WoW view, keeping Sun-Sat weekly bucketing only for the 60-day trend (where stable weekly denominators still matter).

• bootstrap-report.ps1 — new -EndDate param (default: today UTC). Computes curStart = curEnd−7d, prevStart = curEnd−14d, and trendEnd = startofweek(curEnd). Stamps the resolved window into the <title>, <div class="meta"> block, and Generated banner so header dates can never drift from what was queried (this is the ADO acceptance criterion). Refuses future end-dates. Collision fingerprint stopped keying off title/meta (which bootstrap now rewrites unconditionally) — uses firstKpi + size ratio only.

• 15 KQL templates — 12 primary/WoW queries switched to half-open [prevStart, curEnd) with wk = iff(EventInfo_Time >= curStart, curStart, prevStart); downstream JS helpers unchanged because wk still sorts lexicographically. The 3 60-day-trend queries keep startofweek() bucketing; only <END> renamed to <TREND_END> with clarified startofweek(today)-exclusive semantics. Placeholder audit is now consistent across all 15 files.

• SKILL.md — rewrote Inputs to confirm §1: do NOT ask the user for a date by default; just run bootstrap. Override via -EndDate only when the user explicitly requests a non-default window. Filename convention <sunday> → <end-date> throughout. All KQL placeholder examples switched to the rolling-window form.

• validate-report.ps1 — new check Migrate to AndroidX #11 asserts the meta-line "Last 7 days" dates are self-consistent with the filename's end-date. Catches any hand-edit that broke the auto-stamp, or a stale template stub being published fresh.

• Docs — template-readme.md § Date fields rewritten (bootstrap owns every date field; author never edits them). kusto-cheatsheet.md gains § 3 Rolling 7-day WoW window with the canonical let curStart/curEnd/prevStart template; the pre-existing startofweek()-alignment section is now scoped explicitly to the 60-day trend.

Breaking changes (intentional)

• bootstrap-report.ps1: -ReportingSunday parameter removed; use -EndDate. The old parameter name encoded the exact semantic mismatch that caused the bug — keeping it would have preserved the footgun.
• Output filename: oncall-wow-report-<end-date>.html (was <sunday>.html). Old on-disk reports are unaffected.

Verification

• Run on 2026-07-09 → correctly resolves to [2026-07-02, 2026-07-09).
• -EndDate 2026-07-02 → [2026-06-25, 2026-07-02).
• -EndDate 2027-01-01 → refused (future-date guard).
• Same-day re-bootstrap of unfilled stub → silently overwrites.
• Same-day re-bootstrap after author populated KPI → HARD HALT exit 2 unless -Force.
• Validator check Migrate to AndroidX #11 passes on fresh stub, FAILs on a hand-mutated meta line.
• Placeholder audit across all 15 rewritten .kql files — no stale <START>/<END>/<CURR_*>/<PREV_WEEK> references remain.

[github-actions]

✅ Work item link check complete. Description contains link AB#3683194 to an Azure Boards work item.

[github-actions]

✅ Work item link check complete. Description contains link AB#3683194 to an Azure Boards work item.

[github-actions]

❌ Invalid work item number: AB#3683194

##. Work item number must be a valid integer.

Click here to learn more.

[shahzaibj]

Seems like a ton of changes for a simple PR.

[Copilot]

Pull request overview

Updates the OCE on-call telemetry report workflow to default to a rolling 7-day window ending at start-of-day UTC “today” (with optional -EndDate override), fixing the prior Sun→Sat default that could select a future/partial window and miss the most recent days.

Changes:

• Replaces -ReportingSunday with -EndDate and makes bootstrap stamp the resolved window into the report header.
• Updates WoW/primary KQL templates to use half-open rolling windows while keeping startofweek() bucketing for 60-day trends (with partial-week display vs classification split).
• Adds/updates validation + documentation to align authors on the new default semantics and placeholder conventions.

Reviewed changes

Copilot reviewed 26 out of 26 changed files in this pull request and generated 5 comments.

Show a summary per file

File	Description
.github/skills/oncall-weekly-telemetry-report/SKILL.md	Updates skill instructions to run bootstrap without prompting and to use rolling-window defaults/terminology.
.github/skills/oncall-weekly-telemetry-report/assets/templates/template-readme.md	Documents bootstrap-owned header dates and the new rolling-window semantics.
.github/skills/oncall-weekly-telemetry-report/assets/templates/report-template.html	Adjusts template prose to reflect rolling window + updated 60-day trend wording.
.github/skills/oncall-weekly-telemetry-report/assets/scripts/visual-smoke.ps1	Updates examples/output naming to <end-date> convention.
.github/skills/oncall-weekly-telemetry-report/assets/scripts/validate-report.ps1	Adds rolling-window header ↔ filename consistency check.
.github/skills/oncall-weekly-telemetry-report/assets/scripts/summarize-attribution.js	Clarifies expected wk semantics for rolling-window buckets.
.github/skills/oncall-weekly-telemetry-report/assets/scripts/bucket-trends.js	Supports “include partial end week in chart but exclude from delta classification” via --include-partial-end.
.github/skills/oncall-weekly-telemetry-report/assets/scripts/bootstrap-report.ps1	Implements rolling window resolution, future-date guard, header stamping, and updated collision detection.
.github/skills/oncall-weekly-telemetry-report/assets/scripts/agg.js	Updates docs/schema assumptions for rolling-window wk bucketing.
.github/skills/oncall-weekly-telemetry-report/assets/queries/wow-table-sparkline-series.kql	Keeps per-row WoW sparklines on 8 complete weeks using <SPARK_START>/<SPARK_END>.
.github/skills/oncall-weekly-telemetry-report/assets/queries/wow-movers.kql	Converts WoW movers query to rolling-window placeholders and half-open bounds.
.github/skills/oncall-weekly-telemetry-report/assets/queries/type-subcode-decomposition.kql	Converts type sub-code decomposition to rolling-window placeholders and two-bucket wk.
.github/skills/oncall-weekly-telemetry-report/assets/queries/reliability-auth-only.kql	Converts auth-only denominator query to rolling-window placeholders and two-bucket wk.
.github/skills/oncall-weekly-telemetry-report/assets/queries/README.md	Re-documents placeholder conventions (CUR_*, TREND_*, SPARK_*) and partial-week rules.
.github/skills/oncall-weekly-telemetry-report/assets/queries/os-version-slice.kql	Converts OS slice query to rolling-window placeholders and half-open bounds.
.github/skills/oncall-weekly-telemetry-report/assets/queries/latency.kql	Converts latency query to rolling-window placeholders and two-bucket wk.
.github/skills/oncall-weekly-telemetry-report/assets/queries/error-message-and-location.kql	Converts throw-site/message query to rolling-window placeholders and half-open bounds.
.github/skills/oncall-weekly-telemetry-report/assets/queries/broker-version-share.kql	Converts broker version share query to rolling-window placeholders and two-bucket wk.
.github/skills/oncall-weekly-telemetry-report/assets/queries/broker-version-share-wow.kql	Converts WoW snapshot query to rolling-window placeholders and half-open bounds.
.github/skills/oncall-weekly-telemetry-report/assets/queries/attr-union-by-dim.kql	Updates union attribution query to rolling-window placeholders and wk=iff(...) bucketing.
.github/skills/oncall-weekly-telemetry-report/assets/queries/attr-types-by-dim.kql	Updates per-dim type attribution query to rolling-window placeholders and wk=iff(...).
.github/skills/oncall-weekly-telemetry-report/assets/queries/attr-codes-by-dim.kql	Updates per-dim code attribution query to rolling-window placeholders and wk=iff(...).
.github/skills/oncall-weekly-telemetry-report/assets/queries/app-share.kql	Updates traffic/app share query to rolling-window placeholders and two-bucket wk.
.github/skills/oncall-weekly-telemetry-report/assets/queries/60d-trend-types.kql	Updates 60-day trend types query to <TREND_START>/<TREND_END> semantics (no source-side partial-week filter).
.github/skills/oncall-weekly-telemetry-report/assets/queries/60d-trend-codes.kql	Updates 60-day trend codes query to <TREND_START>/<TREND_END> semantics (partial-week handled in JS).
.github/skills/oncall-weekly-telemetry-report/assets/docs/kusto-cheatsheet.md	Adds canonical rolling-window snippet + clarifies where startofweek() still applies.

[shahzaibj]

At the time of invoking the skill, if a user explicitly provides a different time window then will the skill handle that?

[shahzaibj]

On the file count: defaulting to a rolling window forces every query that was Sunday-anchored or hardcoded onto the shared CUR_* / TREND_* / SPARK_* placeholders, so the resolved window flows through the whole pipeline consistently. Most of the ~26 files are mechanical 1:1 placeholder swaps, not logic changes -- the actual behavior change lives in bootstrap-report.ps1 (window computation + header stamping) and the 60d-trend query.

Happy to split the pure-doc files (queries/README.md, kusto-cheatsheet.md, template-readme.md) into a follow-up PR if you'd prefer a smaller diff here. The KQL + script edits are load-bearing for the default to work, so those need to stay together.

[shahzaibj]

Verified. The only override is -EndDate, which shifts the end of the window; the 7-day primary span, the 7-day baseline, and the 60-day trend all move rigidly with it (future dates are rejected by the guard). There is no custom start-date and no arbitrary-span flag -- "last 30 days" or "Jun 1 -> Jun 20" is not expressible via a single parameter today; that still requires hand-editing the KQL <CUR_START> / <CUR_END> placeholders.

I chose to document this limitation explicitly in SKILL.md (Inputs section) rather than silently emit a 7-day report when a user asks for something else, so we surface it up front. If we want true custom spans, a -Days N (or -StartDate) override is a clean follow-up -- I kept it out of this PR to stay scoped.

[Copilot]

Pull request overview

Copilot reviewed 26 out of 26 changed files in this pull request and generated 4 comments.

[Copilot]

Pull request overview

Copilot reviewed 26 out of 26 changed files in this pull request and generated 1 comment.

Comments suppressed due to low confidence (1)

.github/skills/oncall-weekly-telemetry-report/assets/templates/report-template.html:566

• The 60-day trend methodology text still hardcodes an example-specific partial-week date ("Dropped the partial start week (Mar 1)"). Since header dates are now auto-stamped per-run and the 60-day window is rolling, this hardcoded date will become stale/misleading in every newly bootstrapped report unless authors remember to hunt it down manually.

<h2 id="trend60d">📈 60-Day Trend Analysis <span class="count">— literal last 60 days ending today; final bar = current partial week</span></h2>

<div class="glossary">
  <strong>Methodology:</strong> Pulled all error codes from the <code>ErrorStats</code> view over the last 9 weeks. Dropped the partial start week (Mar 1). Kept all codes whose <strong>peak weekly device count ≥ 10 K</strong>. Bucketed each 8-week series by <code>delta = (last − first) / first</code>:
  <strong>regression</strong> if delta &gt; +15% and trajectory is monotonic-ish; <strong>ephemeral spike</strong> if peak ≥ 3× mean of surrounding weeks; <strong>improvement</strong> if delta &lt; −15%; <strong>flat</strong> otherwise. Every code in the regression list gets a spike-attribution card below.

[fadidurah]

Thanks for taking care of this! But 26 files seems huge for the kind of change i suggested. Perhaps the skill is a bit too complex, i feel this makes upkeep a little bit more difficult in the future. have we considered making this less verbose or trimming it down a bit? If we ever want to change something else about this skill or similar skills we have, will it also be this volume of file changes?

[shahzaibj]

@fadidurah

Fair callout — here's the actual breakdown of the 26:

• 1 file is the real change: bootstrap-report.ps1 — the window math, header stamp, filename, and future-date guard. ~75% of the net new lines live here.
• 15 are .kql query files, and they're why the count looks big. The old window was hardcoded / Sunday-anchored independently inside each query. This PR routes all 15 onto shared, bootstrap-resolved placeholders (<CUR_START> / <CUR_END> / <PREV_START> / <TREND_END>). Each edit is the same ~12-line 1:1 swap, not new logic — but every query has to move or that section keeps querying the stale window.
• 4 scripts (bucket-trends.js, agg.js, summarize-attribution.js, validate-report.ps1) plumb/enforce the new contract; visual-smoke.ps1 is a 1-line touch.
• 5 are docs + the HTML template (SKILL.md, cheatsheet, queries README, template-readme, report-template.html).

On your forward-looking question — no, future changes won't be this heavy. This PR pays a one-time centralization cost: the window is now a single control point. Changing it again (rolling 14-day, month-to-date, whatever) is an edit to the resolver in bootstrap-report.ps1 only — the 15 queries already consume whatever it resolves (I verified all 15 read the shared placeholders). So this is the change that removes the 15-file blast radius for next time, not one that repeats it.

The skill's overall size is real, but it's inherent to what it does — a validated, self-contained HTML report across ~15 telemetry dimensions — not verbosity this PR introduced. If you'd like a smaller review surface here, I'm happy to pull the 4 pure-doc files into a follow-up doc-only PR so this one is code + template only. Say the word.