PERF: speed up strftime for common format directives#64609
PERF: speed up strftime for common format directives#64609mroeschke merged 2 commits intopandas-dev:mainfrom
Conversation
2ddad72 to
762c01c
Compare
Possible follow-up: C snprintf pathIf we need more speed, there's a straightforward follow-up that gets the custom-format path close to the hardcoded f-string performance. Instead of On my machine (10.5M elements,
The implementation is simple — always pass all 7 struct fields in fixed order, and the template references them by position — but POSIX positional specifiers aren't available on Windows, so it would need a runtime check ( |
6457039 to
2dc41a2
Compare
d7dd9db to
6836ab8
Compare
| return None | ||
| result.append(replacement) | ||
| i += 2 | ||
| elif fmt[i] in ("{", "}"): |
There was a problem hiding this comment.
We support replacement brackets in a format string like `format="%Y-{1:02d}"? Do we have tests for this case?
For formats composed of common directives (%Y, %m, %d, %H, %M, %S, %f), parse the format string once into a str.format() template and format directly from npy_datetimestruct fields, avoiding per-element Timestamp creation. For tz-aware data, converts UTC to local time via Localizer before extracting struct fields. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
c2216c9 to
42f745c
Compare
|
Thanks @jbrockmendel |
Summary
%Y,%m,%d,%H,%M,%S,%f), parse the format string once into astr.format()template and format directly fromnpy_datetimestructfields, avoiding per-elementTimestampcreation.Localizer.utc_val_to_local_valbefore extracting struct fields (works because_convert_strftime_formatreturnsNonefor formats containing%z/%Z).%A,%B,%z,%Z, etc.) fall back to the existing per-elementTimestamp.strftime()path.Benchmarks (10.5M elements,
%Y/%m/%dT%H:%M:%S)Test plan
Timestamp.strftime()for multiple formats%A,%Z)🤖 Generated with Claude Code