BUG: fix float-to-int64 OOB handling on ARM in to_datetime/to_timedelta#64643
Merged
mroeschke merged 2 commits intopandas-dev:mainfrom Mar 20, 2026
Merged
Conversation
On ARM, float-to-int64 overflow saturates to INT64_MAX instead of wrapping to INT64_MIN (as on x86). This caused the integer shortcut in _to_datetime_with_unit and sequence_to_td64ns to misfire for OOB float values like float(2**63), silently producing incorrect results. Fix by adding an in_int64_range guard to the shortcut condition, replacing the separate OOB pre-check added in GH#64619. This also removes duplicated OOB logic between Python and Cython, and fixes a latent bug in the timedelta path for non-ns units. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
mroeschke
reviewed
Mar 20, 2026
| # instead of wrapping, which makes the data == int_data | ||
| # check pass incorrectly for OOB values like float(2**63). | ||
| # Exclude values outside the int64 domain from the check. | ||
| in_int64_range = (data >= np.float64(-(2**63))) & (data < np.float64(2**63)) |
Member
There was a problem hiding this comment.
Could we use np.float64(np.iinfo(np.int64).max/min) instead of hardcoding the 2**63?
mroeschke
reviewed
Mar 20, 2026
| # instead of wrapping, which makes the arg == int_values | ||
| # check pass incorrectly for OOB values like float(2**63). | ||
| # Exclude values outside the int64 domain from the check. | ||
| in_int64_range = (arg >= np.float64(-(2**63))) & (arg < np.float64(2**63)) |
Member
There was a problem hiding this comment.
Same here (and ideally the test)
Member
Author
There was a problem hiding this comment.
updated here, forgot to do it in the test. for worktree-related reasons updating again is a bit inconvenient ATM
…e check Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
mroeschke
approved these changes
Mar 20, 2026
Member
|
Thanks @jbrockmendel |
Sharl0tteIsTaken
added a commit
to Sharl0tteIsTaken/pandas
that referenced
this pull request
Mar 22, 2026
…h-origin * upstream/main: (199 commits) DEPR: Change assert_index_equal(exact=) default from equiv to True (pandas-dev#64542) DOC: Fix invalid Sphinx cross-references in older whatsnew RST files (pandas-dev#64459) TYP: Allow type[csv.Dialect] to be passed to read_csv() (pandas-dev#64559) TST: add regression tests for Needs Tests issues (pandas-dev#64745) TST: add regression test for loc setitem with DatetimeIndex and str column name (pandas-dev#64743) BUG: fix convert_dtypes dropping values from sliced mixed-dtype DataFrames (pandas-dev#64712) BUG: Fix get_level_values for boolean, NA-like, and integer index names (pandas-dev#63514) BUG: Fix 2D ExtensionArray handling in construction and CSV serialization (pandas-dev#64634) BUG: fix float-to-int64 OOB handling on ARM in to_datetime/to_timedelta (pandas-dev#64643) CLN: Make _CustomBusinessMonth.cbday_roll and month_roll private (pandas-dev#64739) DEPR: ExcelFile.parse (pandas-dev#64724) DEPR: unnecessary *args/**kwargs in groupby ops (pandas-dev#64720) DIST: include vendored licenses in wheel (pandas-dev#64709) DOC: Fix numpydoc validation warnings for DataFrame.plot.__call__ (pandas-dev#64736) CLN: Make offsets.BusinessHour.next_bday private (pandas-dev#64715) CLN: make _CustomBusinessMonth.cbday_roll private (pandas-dev#64734) ENH: Support axis parameter in BaseMaskedArray.any/all for 2D (pandas-dev#64510) PERF: Micro-optimizations in C extension source files (pandas-dev#64669) DEPR: Deprecate xlrd and pyxlsb Excel engines in favor of calamine (pandas-dev#64640) DOC: fix incomplete sentence in Styler docstring (pandas-dev#64682) ...
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Follow-up to #64619. On ARM, float-to-int64 overflow saturates to
INT64_MAXinstead of wrapping toINT64_MIN(as on x86). This caused the integer shortcut in_to_datetime_with_unitandsequence_to_td64nsto misfire for OOB float values likefloat(2**63), becausenp.float64(2**63) == np.int64(INT64_MAX)evaluates toTrueon ARM (INT64_MAX rounds up to 2**63 in float64).#64619 fixed the datetime case with a separate OOB pre-check in Python, but left a latent bug in the timedelta path for non-ns units. This PR replaces the pre-check with a tighter fix: an
in_int64_rangeguard on the shortcut condition itself, applied consistently to both paths.Changes:
datetimes.py: Replace the 12-line OOB pre-check with a 4-linein_int64_rangeguard on the integer shortcut condition. OOB values now fall through tocast_from_unit_vectorizedwhich already has its own OOB check, and the existingtry/excepthandles theerrorsparameter. Removes the now-unusediNaTimport.timedeltas.py: Add the samein_int64_rangeguard to the timedelta integer shortcut, fixing the latent ARM bug for non-ns units.test_to_timedelta.py: Addtest_float_to_timedelta_raise_oob_non_nscovering the timedelta shortcut path withunit="s".Test plan
test_float_to_datetime_raise_oob_nspassestest_float_to_timedelta_raise_oob_nspassestest_float_to_timedelta_raise_oob_non_nspassestest_to_datetime.pyandtest_to_timedelta.pysuites pass (1024 passed)🤖 Generated with Claude Code