gh-149277: Fix error position for invalid numeric literals

[anujbharambe]

Summary

Fix the SyntaxError caret position for invalid numeric literals. Previously,
the caret pointed at the last valid digit instead of the first invalid character.

For example, 0x9g now correctly shows:
0x9g
^
SyntaxError: invalid hexadecimal literal

Instead of the previous incorrect output:
0x9g
^
SyntaxError: invalid hexadecimal literal

The issue was a spurious tok_backup(tok, c) call in verify_end_of_number()
in the error branch. Since _PyTokenizer_syntaxerror computes the column offset
from tok->cur, backing up one character caused the caret to point at the
preceding (valid) character. Removing the backup keeps tok->cur at the correct
position.

Fixes #149277

• Issue: Error display for invalid literal is incorrect #149277

[picnixz]

Please add tests.

[bedevere-app]

A Python core developer has requested some changes be made to your pull request before we can consider merging it. If you could please address their requests along with any other requests in other reviews from core developers that would be appreciated.

Once you have made the requested changes, please leave a comment on this pull request containing the phrase I have made the requested changes; please review again. I will then notify any core developers who have left a review that you're ready for them to take another look at this pull request.

[read-the-docs-community]

Documentation build overview

📚 cpython-previews | 🛠️ Build #32571271 | 📁 Comparing 20996e1 against main (ff422bd)

  🔍 Preview build  

18 files changed · ± 18 modified

± Modified

• c-api/exceptions.html
• deprecations/index.html
• deprecations/pending-removal-in-3.17.html
• deprecations/pending-removal-in-3.19.html
• howto/perf_profiling.html
• library/debug.html
• library/imaplib.html
• library/inspect.html
• library/timeit.html
• library/webbrowser.html
• and 8 more...

[anujbharambe]

I have made the requested changes; please review again.

Added test_end_of_numerical_literals_offset in Lib/test/test_grammar.py that verifies SyntaxError.offset points at the first invalid character for various invalid numeric literal cases (hex, binary, octal, decimal, float, scientific notation, and imaginary literals).

[bedevere-app]

Thanks for making the requested changes!

@picnixz: please review the changes made to this pull request.