Skip to content

Add Grammar-aware constrained decoding#183

Merged
Ki-Seki merged 3 commits into
mainfrom
Ki-Seki/issue165
May 5, 2026
Merged

Add Grammar-aware constrained decoding#183
Ki-Seki merged 3 commits into
mainfrom
Ki-Seki/issue165

Conversation

@Ki-Seki
Copy link
Copy Markdown
Member

@Ki-Seki Ki-Seki commented May 5, 2026

Fixes #165

Copilot AI review requested due to automatic review settings May 5, 2026 08:07
@codecov
Copy link
Copy Markdown

codecov Bot commented May 5, 2026

Codecov Report

✅ All modified and coverable lines are covered by tests.

📢 Thoughts on this report? Let us know!

Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR adds an include_grammar option to model calls so that GIM queries can optionally include tag-level grammar information (currently the regex attribute) directly in the prompt sent to LLM backends, addressing the need for grammar visibility described in #165.

Changes:

  • Add include_grammar: bool = False to OpenAI and vLLM model call APIs and plumb it through the shared _call / _acall path.
  • Extend prompt construction (get_outlines_model_input) to use a new Query.to_string_with_grammar() representation when include_grammar=True.
  • Add/adjust tests to cover the string transformation behavior for include_grammar.

Reviewed changes

Copilot reviewed 8 out of 8 changed files in this pull request and generated 3 comments.

Show a summary per file
File Description
src/gimkit/contexts.py Adds Query.to_string_with_grammar() to include regex in serialized tags.
src/gimkit/models/utils.py Adds include_grammar plumbing to optionally serialize queries with grammar for model input.
src/gimkit/models/base.py Threads include_grammar through shared sync/async call helpers.
src/gimkit/models/openai.py Exposes include_grammar on OpenAI wrappers and forwards it into shared call path.
src/gimkit/models/vllm.py Exposes include_grammar on vLLM wrappers and forwards it into shared call path.
tests/models/test_utils.py Adds explicit assertions for include_grammar affecting prompt string generation.
tests/models/test_openai.py Updates usage to pass include_grammar=True (currently without asserting behavior).
tests/models/test_vllm.py Updates usage to pass include_grammar=True (currently without asserting behavior).

Comment thread src/gimkit/models/base.py
Comment thread tests/models/test_openai.py
Comment thread tests/models/test_vllm.py
@Ki-Seki Ki-Seki merged commit a842e3b into main May 5, 2026
15 checks passed
@Ki-Seki Ki-Seki deleted the Ki-Seki/issue165 branch May 5, 2026 08:20
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

feat: grammar-aware constrained decoding

2 participants