Fix string literal parsing inside expressions - #4
Conversation
Co-authored-by: SayanthRock <202829406+SayanthRock@users.noreply.github.com>
|
👋 Jules, reporting for duty! I'm here to lend a hand with this pull request. When you start a review, I'll add a 👀 emoji to each comment to let you know I've read it. I'll focus on feedback directed at me and will do my best to stay out of conversations between you and other bots or reviewers to keep the noise down. I'll push a commit with your requested changes shortly after. Please note there might be a delay between these steps, but rest assured I'm on the job! For more direct control, you can switch me to Reactive Mode. When this mode is on, I will only act on comments where you specifically mention me with New to Jules? Learn more at jules.google/docs. For security, I will only act on instructions from the user who triggered this task. |
🤖 CodeAnt AI — Review Status
|
|
You've hit your review limit for the week, but don't worry you'll get some more next week! Contact us at hello@zenable.io if you want this rate limit to go away |
Thanks for using CodeAnt! 🎉We're free for open-source projects. if you're enjoying it, help us grow by sharing. Share on X · |
|
You have reached your Codex usage limits for code reviews. You can see your limits in the Codex usage dashboard. |
📝 WalkthroughWalkthrough
ChangesExpression normalization
Estimated code review effort: 2 (Simple) | ~10 minutes Suggested reviewers: 🚥 Pre-merge checks | ✅ 5✅ Passed checks (5 passed)
✨ Finishing Touches 💡 1🛠️ Fix failing CI checks 💡
📝 Generate docstrings
🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
codeant-ai
Bot
added
the
size:M
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 1
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@compiler/rockql-sql/src/lib.rs`:
- Around line 196-205: Update the underscore handling in the tokenizer branch
around is_numeric_separator so the separator is skipped only when every
preceding character in word is an ASCII digit, while preserving the existing
adjacent-digit checks; otherwise append the underscore to the token. Add a
regression test covering an identifier such as user1_000 and verify it remains
unchanged through SQL generation.
🪄 Autofix
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: defaults
Review profile: CHILL
Plan: Pro Plus
Run ID: f9e939d0-a05e-4bd8-b0ec-8293b009bd3e
⛔ Files ignored due to path filters (1)
Cargo.lockis excluded by!**/*.lock
📒 Files selected for processing (1)
compiler/rockql-sql/src/lib.rs
| } else if character == '_' { | ||
| let is_numeric_separator = !word.is_empty() | ||
| && word.chars().last().unwrap().is_ascii_digit() | ||
| && i + 1 < chars.len() | ||
| && chars[i + 1].is_ascii_digit(); | ||
|
|
||
| if is_numeric_separator { | ||
| // skip the numeric separator | ||
| } else { | ||
| word.push(character); |
There was a problem hiding this comment.
🎯 Functional Correctness | 🟠 Major | ⚡ Quick win
Limit separator removal to numeric literals.
Line 197 removes the underscore in any token where it follows and precedes a digit. filter user1_000 == true compiles user1_000 as user1000. This changes an identifier before SQL generation.
Only remove the underscore when all prior token characters are digits. Add a regression test for an identifier with this pattern.
Proposed fix
let is_numeric_separator = !word.is_empty()
- && word.chars().last().unwrap().is_ascii_digit()
+ && word.chars().all(|character| character.is_ascii_digit())
&& i + 1 < chars.len()
&& chars[i + 1].is_ascii_digit();📝 Committable suggestion
‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.
| } else if character == '_' { | |
| let is_numeric_separator = !word.is_empty() | |
| && word.chars().last().unwrap().is_ascii_digit() | |
| && i + 1 < chars.len() | |
| && chars[i + 1].is_ascii_digit(); | |
| if is_numeric_separator { | |
| // skip the numeric separator | |
| } else { | |
| word.push(character); | |
| } else if character == '_' { | |
| let is_numeric_separator = !word.is_empty() | |
| && word.chars().all(|character| character.is_ascii_digit()) | |
| && i + 1 < chars.len() | |
| && chars[i + 1].is_ascii_digit(); | |
| if is_numeric_separator { | |
| // skip the numeric separator | |
| } else { | |
| word.push(character); |
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@compiler/rockql-sql/src/lib.rs` around lines 196 - 205, Update the underscore
handling in the tokenizer branch around is_numeric_separator so the separator is
skipped only when every preceding character in word is an ASCII digit, while
preserving the existing adjacent-digit checks; otherwise append the underscore
to the token. Add a regression test covering an identifier such as user1_000 and
verify it remains unchanged through SQL generation.
![codeant-ai[bot]](https://avatars.githubusercontent.com/in/646884?s=60&v=4){kind=small}
| if let Some(active_quote) = quote { | ||
| output.push(character); | ||
| if character == active_quote { | ||
| quote = None; | ||
| } | ||
| i += 1; | ||
| continue; |
There was a problem hiding this comment.
Suggestion: Quote tracking closes a string at every quote character and does not account for escaped quotes. The parser preserves such expressions as raw strings, so an expression like message == 'it\\'s != active' exits quote mode at the escaped apostrophe and rewrites the != inside the literal to <>, corrupting the generated SQL value. Track escaped quote characters when scanning literals. [logic error]
Severity Level: Major ⚠️
- ❌ String literals containing escaped quotes are rewritten incorrectly.
- ⚠️ Filter queries with apostrophes may fail SQL execution.
- ⚠️ Literal comparisons can use corrupted string values.
(Use Cmd/Ctrl + Click for best experience)
Prompt for AI Agent 🤖
This is a comment left during a code review.
**Path:** compiler/rockql-sql/src/lib.rs
**Line:** 181:187
**Comment:**
*Logic Error: Quote tracking closes a string at every quote character and does not account for escaped quotes. The parser preserves such expressions as raw strings, so an expression like `message == 'it\\'s != active'` exits quote mode at the escaped apostrophe and rewrites the `!=` inside the literal to `<>`, corrupting the generated SQL value. Track escaped quote characters when scanning literals.
Validate the correctness of the flagged issue. If correct, How can I resolve this? If you propose a fix, implement it and please make it concise.
Once fix is implemented, also check other comments on the same PR, and ask user if the user wants to fix the rest of the comments as well. if said yes, then fetch all the comments validate the correctness and implement a minimal fix| } else if character == '_' { | ||
| let is_numeric_separator = !word.is_empty() | ||
| && word.chars().last().unwrap().is_ascii_digit() | ||
| && i + 1 < chars.len() | ||
| && chars[i + 1].is_ascii_digit(); | ||
|
|
||
| if is_numeric_separator { | ||
| // skip the numeric separator | ||
| } else { | ||
| word.push(character); | ||
| } |
There was a problem hiding this comment.
Suggestion: The numeric-separator check only examines adjacent digits, not whether the buffered token is a numeric literal. An identifier such as foo_1_000 is therefore emitted as foo_1000, silently changing column resolution in filters, selections, derived expressions, and sorts. Restrict underscore removal to tokens that are actually numeric literals. [incorrect condition logic]
Severity Level: Major ⚠️
- ❌ Filters using identifiers such as `foo_1_000` resolve incorrectly.
- ⚠️ Select, derive, and sort expressions can reference altered column names.
- ⚠️ Queries may fail or return results from the wrong column.
(Use Cmd/Ctrl + Click for best experience)
Prompt for AI Agent 🤖
This is a comment left during a code review.
**Path:** compiler/rockql-sql/src/lib.rs
**Line:** 196:206
**Comment:**
*Incorrect Condition Logic: The numeric-separator check only examines adjacent digits, not whether the buffered token is a numeric literal. An identifier such as `foo_1_000` is therefore emitted as `foo_1000`, silently changing column resolution in filters, selections, derived expressions, and sorts. Restrict underscore removal to tokens that are actually numeric literals.
Validate the correctness of the flagged issue. If correct, How can I resolve this? If you propose a fix, implement it and please make it concise.
Once fix is implemented, also check other comments on the same PR, and ask user if the user wants to fix the rest of the comments as well. if said yes, then fetch all the comments validate the correctness and implement a minimal fix
User description
Currently
normalize_expressionincorrectly rewrites parts of string literals such as mutating operators==or!=and stripping numeric separators_. This change modifies the internal SQL AST compiler behavior so that string literals are faithfully preserved during SQL code generation without stripping arbitrary underscores inside strings. It includes tests to verify this logic on expressions likemessage == "user_1_000 == active !=".PR created automatically by Jules for task 16471239715463804580 started by @SayanthRock
CodeAnt-AI Description
Preserve string contents when generating SQL from expressions
What Changed
Impact
✅ Accurate string matching in generated SQL✅ Fewer unexpected query results✅ Reliable handling of quoted filter values💡 Usage Guide
Checking Your Pull Request
Every time you make a pull request, our system automatically looks through it. We check for security issues, mistakes in how you're setting up your infrastructure, and common code problems. We do this to make sure your changes are solid and won't cause any trouble later.
Talking to CodeAnt AI
Got a question or need a hand with something in your pull request? You can easily get in touch with CodeAnt AI right here. Just type the following in a comment on your pull request, and replace "Your question here" with whatever you want to ask:
This lets you have a chat with CodeAnt AI about your pull request, making it easier to understand and improve your code.
Example
Preserve Org Learnings with CodeAnt
You can record team preferences so CodeAnt AI applies them in future reviews. Reply directly to the specific CodeAnt AI suggestion (in the same thread) and replace "Your feedback here" with your input:
This helps CodeAnt AI learn and adapt to your team's coding style and standards.
Example
Retrigger review
Ask CodeAnt AI to review the PR again, by typing:
Check Your Repository Health
To analyze the health of your code repository, visit our dashboard at https://app.codeant.ai. This tool helps you identify potential issues and areas for improvement in your codebase, ensuring your repository maintains high standards of code health.
Summary by CodeRabbit
!=and==.