Skip to content

Fix manual runs getting a zero-length data interval across DST#70089

Open
harjothkhara wants to merge 2 commits into
apache:mainfrom
harjothkhara:fix-cron-get-prev-dst
Open

Fix manual runs getting a zero-length data interval across DST#70089
harjothkhara wants to merge 2 commits into
apache:mainfrom
harjothkhara:fix-cron-get-prev-dst

Conversation

@harjothkhara

@harjothkhara harjothkhara commented Jul 19, 2026

Copy link
Copy Markdown

Why

On the two days a year the clocks change, a manually triggered run can get a data interval that is zero length, or that ends after the run time.

With CronDataIntervalTimetable("0 2 * * *", timezone="America/New_York"), a manual run at 2024-03-10 07:30Z:

manual    : 2024-03-10 07:00Z -> 2024-03-10 07:00Z   (zero length)
scheduled : 2024-03-09 07:00Z -> 2024-03-10 07:00Z   (correct)

croniter steps back on naive wall-clock time, but make_aware moves a tick that falls in the spring-forward gap onto the next real instant. When the previous tick is in the gap it lands on the current time, so _get_prev returns the time it was asked about.

That zero-length interval then costs a run. next_dagrun_info reads it as a point-in-time run and skips a period (#66132, correct for the case it was written for):

after the bad interval : 2024-03-11 06:00Z -> 2024-03-12 06:00Z
if the interval is ok  : 2024-03-10 07:00Z -> 2024-03-11 06:00Z

The fall-back side overshoots instead. With 0 1 * * * in America/New_York at 2024-11-03 05:30Z, _get_prev returns 06:00Z — half an hour in the future — so CronTriggerTimetable with run_immediately=True fires a tick that has not happened yet.

Also reproduces in Europe/Zurich, Australia/Sydney and Australia/Lord_Howe.

What

_get_prev now keeps stepping cron.get_prev until the result is strictly earlier than the time asked about. get_prev is strictly decreasing on naive time, so it terminates.

The _covers_every_hour branch is untouched — it uses a wall-clock delta rather than make_aware, so it cannot hit this.

Only values that were already wrong change. Nothing outside a DST transition window is affected.

Testing

Three regression tests, all failing before and passing after:

  • test_cron_infer_manual_data_interval_dst — gap and fall-back, including Australia/Lord_Howe for its 30-minute offset
  • test_cron_next_dagrun_info_after_dst_manual_run — the skipped period
  • test_cron_trigger_run_immediately_does_not_pick_future_runCronTriggerTimetable, which shares the mixin

airflow-core/tests/unit/timetables/ is 246 passed.


Was generative AI tooling used to co-author this PR?
  • Yes (please specify the tool below)

Generated-by: Claude Opus 4.8 following the guidelines

…lier

croniter steps back on naive wall-clock time, but make_aware can map a tick
inside a DST transition forward onto the current instant or later. _get_prev
then returned a "previous" tick that was not strictly earlier, so a manual run
got a zero-length data interval on spring-forward, or one ending after
run_after on fall-back. A zero-length interval also trips the point-in-time
guard in next_dagrun_info and silently skips a scheduled period.
@boring-cyborg

boring-cyborg Bot commented Jul 19, 2026

Copy link
Copy Markdown

Congratulations on your first Pull Request and welcome to the Apache Airflow community! If you have any issues or are unsure about any anything please check our Contributors' Guide
Here are some useful points:

  • Pay attention to the quality of your code (ruff, mypy and type annotations). Our prek-hooks will help you with that.
  • In case of a new feature add useful documentation (in docstrings or in docs/ directory). Adding a new operator? Check this short guide Consider adding an example Dag that shows how users should use it.
  • Consider using Breeze environment for testing locally, it's a heavy docker but it ships with a working Airflow and a lot of integrations.
  • Be patient and persistent. It might take some time to get a review or get the final approval from Committers.
  • Please follow ASF Code of Conduct for all communication including (but not limited to) comments on Pull Requests, Mailing list and Slack.
  • Be sure to read the Airflow Coding style.
  • Always keep your Pull Requests rebased, otherwise your build might fail due to changes not related to your commits.
    Apache Airflow is a community-driven project and together we are making it better 🚀.
    In case of doubts contact the developers at:
    Mailing List: dev@airflow.apache.org
    Slack: https://s.apache.org/airflow-slack

@harjothkhara
harjothkhara marked this pull request as ready for review July 19, 2026 04:55
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant