FEAT add CodeAttackConverter and CodeAttackAttack (closes #1945)#1960
FEAT add CodeAttackConverter and CodeAttackAttack (closes #1945)#1960u7k4rs6 wants to merge 1 commit into
Conversation
Implement CodeAttack (Ren et al., ACL 2024) as a standalone converter
and a PromptSendingAttack subclass following the FlipAttack pattern.
CodeAttackConverter encodes a natural-language prompt word-by-word into
a data-structure initialisation sequence (deque appends, list appends,
or a string assignment) and embeds it in a partial code template that
asks the model to complete the code. Five language variants are
supported: python_stack, python_list, python_string, cpp, go. The
verbose flag selects the _plus template (detailed paragraphs) for the
three Python variants; cpp and go have no plus variant upstream.
CodeAttackAttack wraps the converter in a PromptSendingAttack, prepends
a system prompt that frames the session as code completion, and forwards
language and verbose to the converter. Callers supply a scorer via
AttackScoringConfig as usual.
Files added:
pyrit/prompt_converter/code_attack_converter.py
pyrit/executor/attack/single_turn/code_attack.py
pyrit/datasets/executors/code_attack.yaml
pyrit/datasets/prompt_converters/code_attack_python_stack{,_plus}.yaml
pyrit/datasets/prompt_converters/code_attack_python_list{,_plus}.yaml
pyrit/datasets/prompt_converters/code_attack_python_string{,_plus}.yaml
pyrit/datasets/prompt_converters/code_attack_cpp.yaml
pyrit/datasets/prompt_converters/code_attack_go.yaml
tests/unit/prompt_converter/test_code_attack_converter.py (23 tests)
tests/unit/executor/attack/single_turn/test_code_attack.py (16 tests)
doc/code/executor/attack/code_attack.py
doc/code/executor/attack/code_attack.ipynb
Files modified:
pyrit/prompt_converter/__init__.py
pyrit/executor/attack/single_turn/__init__.py
pyrit/executor/attack/__init__.py
doc/myst.yml
u7k4rs6
left a comment
u7k4rs6
left a comment
There was a problem hiding this comment.
PR Risk Summary
Quality Score: 9/10
Risk Level: low
Merge Recommendation: Safe to merge
Rationale: The changes appear to be focused on adding and refining code attack functionalities and their associated prompt converters. The review found no issues, and the changes are well-contained within the relevant modules. The addition of unit tests further strengthens the quality of this pull request.
| - file: code/executor/attack/4_sequential_attack.ipynb | ||
| - file: code/executor/attack/chunked_request_attack.ipynb | ||
| - file: code/executor/attack/context_compliance_attack.ipynb | ||
| - file: code/executor/attack/code_attack.ipynb |
There was a problem hiding this comment.
we just restructured the attack docs and almost certainly don't want a separate file for it. Can you see if it fits into one of the existing ones (after pulling in latest main)?
There was a problem hiding this comment.
e.g., flip attack is in https://microsoft.github.io/PyRIT/latest/code/executor/single-turn/
|
We need a references.bib update to include the paper. |
| "SingleTurnAttackStrategy", | ||
| "SingleTurnAttackContext", | ||
| "PromptSendingAttack", | ||
| "CodeAttackAttack", |
There was a problem hiding this comment.
That is not an ideal name 😆 CodeAttack is definitely better
| attack_scoring_config: AttackScoringConfig | None = None, | ||
| prompt_normalizer: PromptNormalizer | None = None, | ||
| max_attempts_on_failure: int = 0, | ||
| language: Literal["python_stack", "python_list", "python_string", "cpp", "go"] = "python_stack", |
There was a problem hiding this comment.
We've primarily been using enums for this. See style guide.
| language: Literal["python_stack", "python_list", "python_string", "cpp", "go"] = "python_stack", | ||
| verbose: bool = True, |
There was a problem hiding this comment.
maybe this should just be 1 param that's the template_path and you have an enum with the few provided ones? That way, people can choose from those or provide their own AND it's only 1 param.
| code_converter = PromptConverterConfiguration.from_converters( | ||
| converters=[CodeAttackConverter(language=language, verbose=verbose)] | ||
| ) | ||
| self._request_converters = code_converter + self._request_converters | ||
|
|
||
| system_prompt_path = pathlib.Path(EXECUTOR_SEED_PROMPT_PATH) / "code_attack.yaml" | ||
| system_prompt = SeedPrompt.from_yaml_file(system_prompt_path).value | ||
| self._system_prompt = Message.from_system_prompt(system_prompt=system_prompt) |
There was a problem hiding this comment.
Maybe we don't need a class for this if it's just PromptSendingAttack with a converter? Doesn't seem simpler than just doing that directly? The converter is very useful by itself, of course.
2 participants
Closes #1945.
Summary
Implements CodeAttack (Ren et al., ACL 2024, arXiv:2403.07865), which reformulates a harmful query as a code-completion task. The query is encoded into a data-structure initialization sequence inside a partial code template with a
decode()stub, and the target is asked to complete the code. Because the intent is expressed as a programming task rather than a natural-language request, safety training keyed to natural language triggers less reliably. Black-box, no compute requirements.Two notes for review (deltas from the issue)
Encoding is word-by-word, not character-by-character. The issue described it as char-by-char (from the paper abstract), but the reference implementation (renqibing/CodeAttack) splits on whitespace and hyphens via regex, with character-level only as a fallback for single-token inputs. I matched the reference code. One consequence: separators are normalized on encode (hyphens and runs of whitespace are consumed as delimiters), which is documented in the converter docstring.
Eight templates, not five. The issue scoped five (one per language), but the reference ships eight: the three Python types each have a base and a
_plusverbose variant, and cpp and go have no verbose variant upstream. I included all eight to match the reference. Happy to drop the four_plusfiles if you would rather keep it to five.Design
Follows the FlipConverter + FlipAttack two-class template.
pyrit/prompt_converter/code_attack_converter.py): encodes the prompt into the chosen data-structure operations and renders the code template. Parameters:language(python_stack, python_list, python_string, cpp, go) andverbose(bool, default True, selects the_plusvariant where one exists; intentionally a no-op for cpp and go). Standalone PromptConverter, composes through the normal pipeline.pyrit/executor/attack/single_turn/code_attack.py): subclasses PromptSendingAttack, instantiates the converter and prepends it to the request converters, injects a code-completion system prompt viaprepended_conversation. Reuses existing scorers, no attack-specific scoring.pyrit/datasets/prompt_converters/(matching the CodeChameleon convention) and the system prompt underpyrit/datasets/executors/(matchingflip_attack.yaml).Tests
39 unit tests (23 converter, 16 attack), all passing, following the
test_flip_converter.py/test_flip_attack.pypatterns. Coverage includes per-language template rendering, verbose vs base variants, word-recovery round-trips, empty / special-character / long prompts, converter prepend ordering, system-prompt injection, and scorer invocation through the normal path. Pre-commit (ruff, ty, validate-docs) green.Files
New:
pyrit/prompt_converter/code_attack_converter.pypyrit/executor/attack/single_turn/code_attack.pypyrit/datasets/executors/code_attack.yamlpyrit/datasets/prompt_converters/(code_attack_python_{stack,list,string}{,_plus}.yaml,code_attack_cpp.yaml,code_attack_go.yaml)tests/unit/prompt_converter/test_code_attack_converter.pytests/unit/executor/attack/single_turn/test_code_attack.pydoc/code/executor/attack/code_attack.py(jupytext source, with generated.ipynb)Modified (exports and docs registration):
pyrit/prompt_converter/__init__.pypyrit/executor/attack/single_turn/__init__.pypyrit/executor/attack/__init__.pydoc/myst.ymlChecklist