Add Grammar-aware constrained decoding#183
Merged
Merged
Conversation
Codecov Report✅ All modified and coverable lines are covered by tests. 📢 Thoughts on this report? Let us know! |
Contributor
There was a problem hiding this comment.
Pull request overview
This PR adds an include_grammar option to model calls so that GIM queries can optionally include tag-level grammar information (currently the regex attribute) directly in the prompt sent to LLM backends, addressing the need for grammar visibility described in #165.
Changes:
- Add
include_grammar: bool = Falseto OpenAI and vLLM model call APIs and plumb it through the shared_call/_acallpath. - Extend prompt construction (
get_outlines_model_input) to use a newQuery.to_string_with_grammar()representation wheninclude_grammar=True. - Add/adjust tests to cover the string transformation behavior for
include_grammar.
Reviewed changes
Copilot reviewed 8 out of 8 changed files in this pull request and generated 3 comments.
Show a summary per file
| File | Description |
|---|---|
src/gimkit/contexts.py |
Adds Query.to_string_with_grammar() to include regex in serialized tags. |
src/gimkit/models/utils.py |
Adds include_grammar plumbing to optionally serialize queries with grammar for model input. |
src/gimkit/models/base.py |
Threads include_grammar through shared sync/async call helpers. |
src/gimkit/models/openai.py |
Exposes include_grammar on OpenAI wrappers and forwards it into shared call path. |
src/gimkit/models/vllm.py |
Exposes include_grammar on vLLM wrappers and forwards it into shared call path. |
tests/models/test_utils.py |
Adds explicit assertions for include_grammar affecting prompt string generation. |
tests/models/test_openai.py |
Updates usage to pass include_grammar=True (currently without asserting behavior). |
tests/models/test_vllm.py |
Updates usage to pass include_grammar=True (currently without asserting behavior). |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes #165