fix: filter out bad timestamp issue activities to prevent overflows by epipav · Pull Request #3693 · linuxfoundation/crowd.dev

epipav · 2025-12-12T08:56:20Z

We have 1970-dated issue activities that cause overflows when calculating closedInSeconds and `respondedInSeconds. We now filter these activities out in the issue analysis copy pipe

Note

Low Risk
Low risk SQL filter change that only excludes invalid/pre-1971 timestamps; main impact is potentially dropping a small set of bad events from issue metrics.

Overview
Prevents overflow/invalid duration calculations in the issue analysis Tinybird pipe by filtering out issue opened and closed activities with timestamps before 1971 (toYear(timestamp) >= 1971).

This ensures closedInSeconds/respondedInSeconds are computed only from sane timestamps, at the cost of excluding affected legacy/badly-ingested events from the issues_analyzed copy output.

^{Written by Cursor Bugbot for commit 5702de6. This will update automatically on new commits. Configure here.}

github-actions · 2025-12-12T08:56:29Z

⚠️ Jira Issue Key Missing

Your PR title doesn't contain a Jira issue key. Consider adding it for better traceability.

Example:

feat: add user authentication (CM-123)
feat: add user authentication (IN-123)

Projects:

CM: Community Data Platform
IN: Insights

Please add a Jira issue key to your PR title.

github-actions · 2025-12-12T08:56:34Z

⚠️ Jira Issue Key Missing

Your PR title doesn't contain a Jira issue key. Consider adding it for better traceability.

Example:

feat: add user authentication (CM-123)
feat: add user authentication (IN-123)

Projects:

CM: Community Data Platform
IN: Insights

Please add a Jira issue key to your PR title.

github-actions · 2025-12-12T08:56:41Z

⚠️ Jira Issue Key Missing

Your PR title doesn't contain a Jira issue key. Consider adding it for better traceability.

Example:

feat: add user authentication (CM-123)
feat: add user authentication (IN-123)

Projects:

CM: Community Data Platform
IN: Insights

Please add a Jira issue key to your PR title.

github-actions · 2025-12-12T08:57:09Z

⚠️ Jira Issue Key Missing

Your PR title doesn't contain a Jira issue key. Consider adding it for better traceability.

Example:

feat: add user authentication (CM-123)
feat: add user authentication (IN-123)

Projects:

CM: Community Data Platform
IN: Insights

Please add a Jira issue key to your PR title.

joanagmaia

LGTM ✅

Do you think this might also be an issue for pull requests analysis data? So basically in all widgets where we rely on the timestamps and where we need to make calculations like resolvedAt, mergedAt, shouldn't we always exclude these activities? 🤔
Everywhere else where we have totals we should still include them

joanagmaia · 2026-03-23T17:01:05Z

Hey @epipav I believe this is already in production, can we merge this one?

CLAassistant · 2026-03-23T17:21:59Z

Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.
_{You have signed the CLA already but the status is still pending? Let us recheck it.}

github-actions · 2026-03-23T17:22:09Z

⚠️ Jira Issue Key Missing

Your PR title doesn't contain a Jira issue key. Consider adding it for better traceability.

Example:

feat: add user authentication (CM-123)
feat: add user authentication (IN-123)

Projects:

CM: Community Data Platform
IN: Insights

Please add a Jira issue key to your PR title.

Copilot

Pull request overview

This PR updates the Tinybird issue analysis copy pipe to drop obviously-bad (pre-1971) issue activity timestamps that can lead to incorrect/overflowing duration calculations in downstream metrics.

Changes:

Filters issues-opened and issues-closed activities to toYear(timestamp) >= 1971 to exclude 1970-era bad data.
Normalizes the pipe TYPE value and adjusts whitespace/formatting in the pipe file.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Copilot · 2026-03-23T17:23:55Z

services/libs/tinybird/pipes/issue_analysis_copy_pipe.pipe

@@ -1,5 +1,6 @@
 DESCRIPTION >
-    Compacts activities from same issue into one, keeping necessary information in a single row. Helps to serve issue-wide widgets in the development tab.
+	Compacts activities from same issue into one, keeping necessary information in a single row. Helps to serve issue-wide widgets in the development tab.


Line 2 uses a tab indentation under DESCRIPTION >, while other Tinybird pipes in this repo use spaces (e.g., pull_request_analysis_copy_pipe.pipe). Tabs can make diffs noisy and may be interpreted differently by tooling—please switch this to consistent space indentation.

Suggested change

Compacts activities from same issue into one, keeping necessary information in a single row. Helps to serve issue-wide widgets in the development tab.

Compacts activities from same issue into one, keeping necessary information in a single row. Helps to serve issue-wide widgets in the development tab.

Copilot · 2026-03-23T17:23:55Z

services/libs/tinybird/pipes/issue_analysis_copy_pipe.pipe

    FROM activityRelations_deduplicated_cleaned_bucket_union
-    WHERE type = 'issues-opened'
+    WHERE type = 'issues-opened' AND toYear(timestamp) >= 1971


The new toYear(timestamp) >= 1971 predicate is non-sargable in ClickHouse/Tinybird and can prevent partition/primary-key pruning. Prefer a direct range filter like timestamp >= toDateTime('1971-01-01 00:00:00') (or toDate('1971-01-01') if timestamp is a Date) to keep the filter efficient.

Copilot · 2026-03-23T17:23:56Z

services/libs/tinybird/pipes/issue_analysis_copy_pipe.pipe

    FROM activityRelations_deduplicated_cleaned_bucket_union
-    WHERE type = 'issues-closed' AND sourceParentId != ''
+    WHERE type = 'issues-closed' AND sourceParentId != '' AND toYear(timestamp) >= 1971
    GROUP BY sourceParentId


Same as above: toYear(timestamp) >= 1971 is likely to be less efficient than a direct timestamp range predicate. Consider switching this to timestamp >= toDateTime('1971-01-01 00:00:00') for better query pruning.

Copilot · 2026-03-23T17:23:56Z

services/libs/tinybird/pipes/issue_analysis_copy_pipe.pipe

    LEFT JOIN issues_comment AS comment ON opened.sourceId = comment.sourceParentId

-TYPE COPY
+TYPE copy


TYPE copy deviates from the convention used across the Tinybird pipes in this repo (TYPE COPY). If Tinybird parsing or internal validation is case-sensitive (or if tooling expects the canonical form), this could break deployments—please keep it consistent as TYPE COPY.

Suggested change

TYPE copy

TYPE COPY

cursor

Cursor Bugbot has reviewed your changes and found 1 potential issue.

^{Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, have a team admin enable autofix in the Cursor dashboard.}

services/libs/tinybird/pipes/issue_analysis_copy_pipe.pipe

Signed-off-by: anil <epipav@gmail.com>

…-out

github-actions · 2026-03-23T17:29:12Z

⚠️ Jira Issue Key Missing

Your PR title doesn't contain a Jira issue key. Consider adding it for better traceability.

Example:

feat: add user authentication (CM-123)
feat: add user authentication (IN-123)

Projects:

CM: Community Data Platform
IN: Insights

Please add a Jira issue key to your PR title.

github-actions · 2026-03-23T17:29:41Z

⚠️ Jira Issue Key Missing

Your PR title doesn't contain a Jira issue key. Consider adding it for better traceability.

Example:

feat: add user authentication (CM-123)
feat: add user authentication (IN-123)

Projects:

CM: Community Data Platform
IN: Insights

Please add a Jira issue key to your PR title.

…-out

github-actions · 2026-03-26T15:52:52Z

⚠️ Jira Issue Key Missing

Your PR title doesn't contain a Jira issue key. Consider adding it for better traceability.

Example:

feat: add user authentication (CM-123)
feat: add user authentication (IN-123)

Projects:

CM: Community Data Platform
IN: Insights

Please add a Jira issue key to your PR title.

…-out

github-actions · 2026-03-27T11:19:01Z

⚠️ Jira Issue Key Missing

Your PR title doesn't contain a Jira issue key. Consider adding it for better traceability.

Example:

feat: add user authentication (CM-123)
feat: add user authentication (IN-123)

Projects:

CM: Community Data Platform
IN: Insights

Please add a Jira issue key to your PR title.

Signed-off-by: anil <epipav@gmail.com>

github-actions · 2026-03-27T11:20:38Z

⚠️ Jira Issue Key Missing

Your PR title doesn't contain a Jira issue key. Consider adding it for better traceability.

Example:

feat: add user authentication (CM-123)
feat: add user authentication (IN-123)

Projects:

CM: Community Data Platform
IN: Insights

Please add a Jira issue key to your PR title.

epipav requested a review from joanagmaia December 12, 2025 08:56

joanagmaia previously approved these changes Dec 12, 2025

View reviewed changes

epipav dismissed joanagmaia’s stale review via 090ae14 March 23, 2026 17:21

Copilot AI review requested due to automatic review settings March 23, 2026 17:21

Copilot started reviewing on behalf of epipav March 23, 2026 17:22 View session

Copilot AI reviewed Mar 23, 2026

View reviewed changes

cursor bot reviewed Mar 23, 2026

View reviewed changes

services/libs/tinybird/pipes/issue_analysis_copy_pipe.pipe Outdated Show resolved Hide resolved

fix: filter out bad timestamp issue activities to prevent overflows

9d1ab68

Signed-off-by: anil <epipav@gmail.com>

epipav force-pushed the bugfix/tb-issue-analysis-filtering-old-dates-out branch from 090ae14 to 9d1ab68 Compare March 23, 2026 17:26

epipav added 2 commits March 23, 2026 18:28

chore: formatting

aea6288

Signed-off-by: anil <epipav@gmail.com>

Merge branch 'main' into bugfix/tb-issue-analysis-filtering-old-dates…

444beef

…-out

Merge branch 'main' into bugfix/tb-issue-analysis-filtering-old-dates…

704d7ea

…-out

Merge branch 'main' into bugfix/tb-issue-analysis-filtering-old-dates…

81ca6d3

…-out

chore: formatting

5702de6

Signed-off-by: anil <epipav@gmail.com>

epipav merged commit 31ddfc0 into main Mar 27, 2026
15 checks passed

epipav deleted the bugfix/tb-issue-analysis-filtering-old-dates-out branch March 27, 2026 11:23

	Compacts activities from same issue into one, keeping necessary information in a single row. Helps to serve issue-wide widgets in the development tab.
	Compacts activities from same issue into one, keeping necessary information in a single row. Helps to serve issue-wide widgets in the development tab.

Conversation

epipav commented Dec 12, 2025 • edited by cursor bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions bot commented Dec 12, 2025

Uh oh!

github-actions bot commented Dec 12, 2025

Uh oh!

github-actions bot commented Dec 12, 2025

Uh oh!

github-actions bot commented Dec 12, 2025

Uh oh!

joanagmaia left a comment

Choose a reason for hiding this comment

Uh oh!

joanagmaia commented Mar 23, 2026

Uh oh!

CLAassistant commented Mar 23, 2026

Uh oh!

github-actions bot commented Mar 23, 2026

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Copilot AI Mar 23, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Mar 23, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Mar 23, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Mar 23, 2026

Choose a reason for hiding this comment

Uh oh!

cursor bot left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

github-actions bot commented Mar 23, 2026

Uh oh!

github-actions bot commented Mar 23, 2026

Uh oh!

github-actions bot commented Mar 26, 2026

Uh oh!

github-actions bot commented Mar 27, 2026

Uh oh!

github-actions bot commented Mar 27, 2026

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

epipav commented Dec 12, 2025 •

edited by cursor bot

Loading