Skip to content

Numeric SQL literals with underscores are not correctly parsed #23877

Description

@nuno-faria

Describe the bug

Numeric literals with underscores fail to get parsed, even though they are supported by some dialects. The problem is that the parse_sql_number method uses Rust's parse, which treats underscores as errors (https://doc.rust-lang.org/core/primitive.i64.html#method.from_str).

This is pretty easy to fix, we just need to remove the underscores before parsing. The positive number cases will require an additional string allocation that they didn't need before, but this should be ok since I think it's only used to parse SQL code. However, I want to know whether to apply the fix here or if the underscores should be removed directly in the sqlparser crate.

To Reproduce

> set datafusion.sql_parser.dialect = 'postgres';

> select 1_000;
SQL error: ParserError("Cannot parse 1_000 as f64")

Expected behavior

postgres=# select 1_000;
 ?column?
----------
     1000

Additional context

No response

Metadata

Metadata

Assignees

Labels

bugSomething isn't working

Type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions