From a82ea4394d2c7199da65c66f822f26adb0485d53 Mon Sep 17 00:00:00 2001 From: Yanjun Qi / Jane Date: Fri, 14 Aug 2026 17:46:13 -0400 Subject: [PATCH 1/2] Fix outdated examples, broken links, and stale references across docs MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Reviewed docs/0_get_started/, docs/1start/, and docs/3recipes/ for outdated code examples, broken external links, and stale version/ count references. Each item verified against current code or a live request before fixing: - installation.md: "Python 3.6 or above" and the conda example's `python=3.8` are both stale; setup.py has required >=3.9 since the black-version-upgrade PR. Bumped both to 3.9. - command_line_usage.md: `textattack eval-model` was renamed to `textattack eval` (confirmed against EvalModelCommand.register_subcommand's actual registered name). - quick_api_tour.rst: fixed two cross-references pointing at a nonexistent `transformers.models.wrapper` module (should be `textattack.models.wrappers`), and a malformed `:ref:TextFooler attack` role with no matching target. - what_is_an_adversarial_attack.md: a copy-pasteable CLI example had its `--flag` hyphens autocorrected to em-dashes at some point ("attack — model lstm-mr — num-examples..."), making it non-functional if pasted as-is. Restored `--`. - support.md: pointed contributors at `transformation.rst`, which doesn't exist (it's `docs/api/transformations.rst`). - FAQ.md / attacks4Components.md / attack_recipes_cmd.md: "16 adversarial attacks" is stale (confirmed 21 CLI-registered recipes via ATTACK_RECIPE_NAMES + 3 Python-only multi-lingual recipes = 24); added the missing `a2t`, `leap`, and French/Spanish/Chinese recipe rows to both HTML comparison tables, which listed neither despite both existing in the codebase. - attack_recipes.rst: fixed a stale recipe-numbering gap (17-19 should have been 18-20 after an earlier recipe was inserted above without renumbering everything below it); now sequential 0-23. - attack_recipes_cmd.md: the DeepWordBug example labeled "on DistilBERT trained on the Quora Question Pairs paraphrase identification dataset" used `distilbert-base-uncased-cola` (a *grammatical acceptability* model, unrelated to QQP/paraphrase); fixed to `distilbert-base-cased-qqp`, confirmed to exist on the Hub. - models.md: `github.com/huggingface/nlp` is HuggingFace's old, renamed repo (confirmed it 301-redirects to `github.com/huggingface/datasets`); updated both link text and href. Verified: full clean Sphinx rebuild afterward introduces no new warnings in any of these files (cross-checked against the existing, separately-tracked warning list from PR #837, which touches different files entirely). Co-Authored-By: Claude Sonnet 5 --- docs/0_get_started/command_line_usage.md | 4 +- docs/0_get_started/installation.md | 4 +- docs/0_get_started/quick_api_tour.rst | 4 +- docs/1start/FAQ.md | 2 +- docs/1start/attacks4Components.md | 47 +++++++++++++++++++- docs/1start/support.md | 2 +- docs/1start/what_is_an_adversarial_attack.md | 2 +- docs/3recipes/attack_recipes.rst | 12 ++--- docs/3recipes/attack_recipes_cmd.md | 45 ++++++++++++++++++- docs/3recipes/models.md | 4 +- 10 files changed, 106 insertions(+), 20 deletions(-) diff --git a/docs/0_get_started/command_line_usage.md b/docs/0_get_started/command_line_usage.md index e35af9e8d..2225c71b7 100644 --- a/docs/0_get_started/command_line_usage.md +++ b/docs/0_get_started/command_line_usage.md @@ -102,9 +102,9 @@ Here are some models from transformers that have worked well for us: - `roberta-base` - `xlnet-base-cased` -## Evaluating Models with `textattack eval-model` +## Evaluating Models with `textattack eval` -Any TextAttack-compatible model can be evaluated using `textattack eval-model`. TextAttack-trained models can be evaluated using `textattack eval --num-examples --model /path/to/trained/model/` +Any TextAttack-compatible model can be evaluated using `textattack eval`. TextAttack-trained models can be evaluated using `textattack eval --num-examples --model /path/to/trained/model/` ## Other Commands diff --git a/docs/0_get_started/installation.md b/docs/0_get_started/installation.md index 5a532cb06..dceffb9dd 100644 --- a/docs/0_get_started/installation.md +++ b/docs/0_get_started/installation.md @@ -1,7 +1,7 @@ Installation ============== -To use TextAttack, you must be running Python 3.6 or above. A CUDA-compatible GPU is optional but will greatly improve speed. +To use TextAttack, you must be running Python 3.9 or above. A CUDA-compatible GPU is optional but will greatly improve speed. We recommend installing TextAttack in a virtual environment (check out this [guide](https://packaging.python.org/guides/installing-using-pip-and-virtual-environments/)). @@ -67,7 +67,7 @@ Besides, we highly recommend you to use virtual environment for textattack use, see [information here](https://conda.io/projects/conda/en/latest/user-guide/tasks/manage-environments.html#removing-an-environment). Here is one conda example: ```bash -conda create -n textattackenv python=3.8 +conda create -n textattackenv python=3.9 conda activate textattackenv conda env list ``` diff --git a/docs/0_get_started/quick_api_tour.rst b/docs/0_get_started/quick_api_tour.rst index dc75a2264..4ce792db5 100644 --- a/docs/0_get_started/quick_api_tour.rst +++ b/docs/0_get_started/quick_api_tour.rst @@ -14,14 +14,14 @@ Let us attack a BERT model fine-tuned for sentimental classification task. We ar >>> tokenizer = transformers.AutoTokenizer.from_pretrained("textattack/bert-base-uncased-imdb") -TextAttack requires both the model and the tokenizer to be wrapped by a :class:`~transformers.models.wrapper.ModelWrapper` class that implements the forward pass operation given a list of input texts. For models provided by Transformers library, we can also simply use :class:`~transformers.models.wrapper.HuggingFaceModelWrapper` class which implements both the forward pass and tokenization. +TextAttack requires both the model and the tokenizer to be wrapped by a :class:`~textattack.models.wrappers.ModelWrapper` class that implements the forward pass operation given a list of input texts. For models provided by Transformers library, we can also simply use :class:`~textattack.models.wrappers.HuggingFaceModelWrapper` class which implements both the forward pass and tokenization. .. code-block:: >>> import textattack >>> model_wrapper = textattack.models.wrappers.HuggingFaceModelWrapper(model, tokenizer) -Next, let's build the attack that we want to use. TextAttack provides prebuilt attacks in the form of :class:`~transformers.attack_recipes.AttackRecipe`. For this example, we will use :ref:TextFooler attack +Next, let's build the attack that we want to use. TextAttack provides prebuilt attacks in the form of :class:`~textattack.attack_recipes.AttackRecipe`. For this example, we will use the TextFooler attack. .. code-block:: diff --git a/docs/1start/FAQ.md b/docs/1start/FAQ.md index adc38188c..37adf3432 100644 --- a/docs/1start/FAQ.md +++ b/docs/1start/FAQ.md @@ -145,7 +145,7 @@ The `attack_one` method in an `Attack` takes as input an `AttackedText`, and out We formulate an attack as consisting of four components: a **goal function** which determines if the attack has succeeded, **constraints** defining which perturbations are valid, a **transformation** that generates potential modifications given an input, and a **search method** which traverses through the search space of possible perturbations. The attack attempts to perturb an input text such that the model output fulfills the goal function (i.e., indicating whether the attack is successful) and the perturbation adheres to the set of constraints (e.g., grammar constraint, semantic similarity constraint). A search method is used to find a sequence of transformations that produce a successful adversarial example. -This modular design unifies adversarial attack methods into one system, enables us to easily assemble attacks from the literature while re-using components that are shared across attacks. We provides clean, readable implementations of 16 adversarial attack recipes from the literature (see [our tool paper](https://arxiv.org/abs/2005.05909) and [our benchmark search paper](https://arxiv.org/abs/2009.06368)). For the first time, these attacks can be benchmarked, compared, and analyzed in a standardized setting. +This modular design unifies adversarial attack methods into one system, enables us to easily assemble attacks from the literature while re-using components that are shared across attacks. We provides clean, readable implementations of 24 adversarial attack recipes from the literature (see [our tool paper](https://arxiv.org/abs/2005.05909) and [our benchmark search paper](https://arxiv.org/abs/2009.06368)). For the first time, these attacks can be benchmarked, compared, and analyzed in a standardized setting. diff --git a/docs/1start/attacks4Components.md b/docs/1start/attacks4Components.md index ca4cb3ce0..6c1275cd5 100644 --- a/docs/1start/attacks4Components.md +++ b/docs/1start/attacks4Components.md @@ -2,7 +2,7 @@ To unify adversarial attack methods into one system, We formulate an attack as consisting of four components: a **goal function** which determines if the attack has succeeded, **constraints** defining which perturbations are valid, a **transformation** that generates potential modifications given an input, and a **search method** which traverses through the search space of possible perturbations. The attack attempts to perturb an input text such that the model output fulfills the goal function (i.e., indicating whether the attack is successful) and the perturbation adheres to the set of constraints (e.g., grammar constraint, semantic similarity constraint). A search method is used to find a sequence of transformations that produce a successful adversarial example. -This modular design enables us to easily assemble attacks from the literature while re-using components that are shared across attacks. TextAttack provides clean, readable implementations of 16 adversarial attacks from the literature. For the first time, these attacks can be benchmarked, compared, and analyzed in a standardized setting. +This modular design enables us to easily assemble attacks from the literature while re-using components that are shared across attacks. TextAttack provides clean, readable implementations of 24 adversarial attacks from the literature. For the first time, these attacks can be benchmarked, compared, and analyzed in a standardized setting. - Two examples showing four components of two SOTA attacks ![two-categorized-attacks](/_static/imgs/intro/01-categorized-attacks.png) @@ -44,7 +44,7 @@ A `SearchMethod` takes as input an initial `GoalFunctionResult` and returns a fi ### Four components in Attack Recipes we have implemented -- TextAttack provides clean, readable implementations of 16 adversarial attacks from the literature. +- TextAttack provides clean, readable implementations of 24 adversarial attacks from the literature. - To run an attack recipe: `textattack attack --recipe [recipe_name]` @@ -62,6 +62,14 @@ A `SearchMethod` takes as input an initial `GoalFunctionResult` and returns a fi
Attacks on classification tasks, like sentiment classification and entailment:
 + +a2t +Untargeted Classification +Part-of-speech match, Max modification rate, SBERT sentence encoding cosine similarity, Word embedding distance +Counter-fitted word embedding swap (or BERT Masked Token Prediction, in the `mlm` variant) +Greedy-WIR (gradient) +Attack tuned for use in adversarial training, from (["Towards Improving Adversarial Training of NLP Models" (Yoo et al., 2021)](https://arxiv.org/abs/2109.00544)) + alzantot Untargeted {Classification, Entailment} @@ -142,6 +150,14 @@ A `SearchMethod` takes as input an initial `GoalFunctionResult` and returns a fi Greedy-WIR Greedy attack with word importance ranking , Reducing the input while maintaining the prediction through word importance ranking (["Pathologies of Neural Models Make Interpretation Difficult" (Feng et al., 2018)](https://arxiv.org/pdf/1804.07781.pdf)) + +leap +Untargeted Classification +Max modification rate, Stopword modification +WordNet-based synonym swap +Particle Swarm Optimization (Levy-flight/adaptive-inertia variant) +Levy-flight/adaptive-inertia variant of the `pso` search method, from (["LEAP: Efficient and Automated Test Method for NLP Software" (Ma et al., 2023)](https://arxiv.org/abs/2308.11284)) + kuleshov Untargeted Classification @@ -223,6 +239,33 @@ A `SearchMethod` takes as input an initial `GoalFunctionResult` and returns a fi Uses imperceptible character-level perturbations including homoglyph substitutions, Unicode reordering, deletions, and invisibles. Based on (["Bad Characters: Imperceptible NLP Attacks" (Boucher et al., 2021)](https://arxiv.org/abs/2106.09898)). +
Multi-lingual attacks on non-English classification models (Python API only, no CLI recipe name -- see #423):
 + + +FrenchRecipe +Untargeted Classification +Repeat modification, Stopword modification (French) +WordNet synonym swap, location swap, name swap (French) +Greedy-WIR +Contextualized-perturbation attack adapted for French NLP models. + + +SpanishRecipe +Untargeted Classification +Repeat modification, Stopword modification (Spanish) +WordNet synonym swap, location swap, name swap (Spanish) +Greedy-WIR +Contextualized-perturbation attack adapted for Spanish NLP models. + + +ChineseRecipe +Untargeted Classification +Repeat modification, Stopword modification (Chinese) +HowNet word swap, Masked-LM word swap, Morphonym & Homophone character swap +Greedy-WIR (weighted-saliency) +Contextualized-perturbation attack adapted for Chinese NLP models. + + diff --git a/docs/1start/support.md b/docs/1start/support.md index e7c1d63d6..b10caa43d 100644 --- a/docs/1start/support.md +++ b/docs/1start/support.md @@ -189,7 +189,7 @@ Follow these steps to start contributing: documentation for a new code file should just be two lines. Our docs will automatically generate from the comments you added to your code. If you're adding an attack recipe, add a reference in `attack_recipes.rst`. - If you're adding a transformation, add a reference in `transformation.rst`, etc. + If you're adding a transformation, add a reference in `docs/api/transformations.rst`, etc. You can build the docs and view the updates using `make docs`. If you're adding a tutorial or something where you want to update the docs multiple diff --git a/docs/1start/what_is_an_adversarial_attack.md b/docs/1start/what_is_an_adversarial_attack.md index 975af4dce..fee2643f2 100644 --- a/docs/1start/what_is_an_adversarial_attack.md +++ b/docs/1start/what_is_an_adversarial_attack.md @@ -53,7 +53,7 @@ Naturally, many have wondered about what adversarial examples for NLP models mig ![Two definitions of NLP adversarial examples](/_static/imgs/intro/mr_aes.png) -*Two different ideas of adversarial examples in NLP. These results were generated using TextAttack on an LSTM trained on the Rotten Tomatoes Movie Review sentiment classification dataset. These are *real* adversarial examples, generated using the DeepWordBug and TextFooler attacks. To generate them yourself, after installing TextAttack, run ‘textattack attack — model lstm-mr — num-examples 1 — recipe RECIPE — num-examples-offset 19’ where RECIPE is ‘deepwordbug’ or ‘textfooler’.* +*Two different ideas of adversarial examples in NLP. These results were generated using TextAttack on an LSTM trained on the Rotten Tomatoes Movie Review sentiment classification dataset. These are *real* adversarial examples, generated using the DeepWordBug and TextFooler attacks. To generate them yourself, after installing TextAttack, run ‘textattack attack --model lstm-mr --num-examples 1 --recipe RECIPE --num-examples-offset 19’ where RECIPE is ‘deepwordbug’ or ‘textfooler’.* Because two text sequences are never indistinguishable, researchers have proposed various alternative definitions for adversarial examples in NLP. We find it useful to group adversarial attacks based on their chosen definitions of adversarial examples. diff --git a/docs/3recipes/attack_recipes.rst b/docs/3recipes/attack_recipes.rst index c87d61083..6c0e5ce37 100644 --- a/docs/3recipes/attack_recipes.rst +++ b/docs/3recipes/attack_recipes.rst @@ -149,8 +149,8 @@ Attacks on classification models Attacks on sequence-to-sequence models ############################################ -17. MORPHEUS (It’s Morphin’ Time! Combating Linguistic Discrimination with Inflectional Perturbations) -18. Seq2Sick (Seq2Sick: Evaluating the Robustness of Sequence-to-Sequence Models with Adversarial Examples) +18. MORPHEUS (It’s Morphin’ Time! Combating Linguistic Discrimination with Inflectional Perturbations) +19. Seq2Sick (Seq2Sick: Evaluating the Robustness of Sequence-to-Sequence Models with Adversarial Examples) .. automodule:: textattack.attack_recipes.morpheus_tan_2020 @@ -166,7 +166,7 @@ Attacks on sequence-to-sequence models General ############################################ -19. BadCharacters (Bad Characters: Imperceptible NLP Attacks) +20. BadCharacters (Bad Characters: Imperceptible NLP Attacks) .. automodule:: textattack.attack_recipes.bad_characters_2021 @@ -181,9 +181,9 @@ TextAttack also includes recipes for non-English classification models, each a language-specific adaptation of a contextualized-perturbation attack (see https://github.com/QData/TextAttack/issues/423): -20. FrenchRecipe -- Attack French Recipe (Contextualized Perturbation for French NLP Adversarial Attack) -21. SpanishRecipe -- Attack Spanish Recipe (Contextualized Perturbation for Spanish NLP Adversarial Attack) -22. ChineseRecipe -- Attack Chinese Recipe (Contextualized Perturbation for Chinese NLP Adversarial Attack) +21. FrenchRecipe -- Attack French Recipe (Contextualized Perturbation for French NLP Adversarial Attack) +22. SpanishRecipe -- Attack Spanish Recipe (Contextualized Perturbation for Spanish NLP Adversarial Attack) +23. ChineseRecipe -- Attack Chinese Recipe (Contextualized Perturbation for Chinese NLP Adversarial Attack) .. automodule:: textattack.attack_recipes.french_recipe diff --git a/docs/3recipes/attack_recipes_cmd.md b/docs/3recipes/attack_recipes_cmd.md index a8e35ab60..3118d26b8 100644 --- a/docs/3recipes/attack_recipes_cmd.md +++ b/docs/3recipes/attack_recipes_cmd.md @@ -39,7 +39,7 @@ textattack attack --recipe textfooler --model bert-base-uncased-mr --num-example _DeepWordBug on DistilBERT trained on the Quora Question Pairs paraphrase identification dataset_: ```bash -textattack attack --model distilbert-base-uncased-cola --recipe deepwordbug --num-examples 100 +textattack attack --model distilbert-base-cased-qqp --recipe deepwordbug --num-examples 100 ``` _Beam search with beam width 4 and word embedding transformation and untargeted goal function on an LSTM_: @@ -73,6 +73,14 @@ To run an attack recipe: `textattack attack --recipe [recipe_name]`
Attacks on classification tasks, like sentiment classification and entailment:
 + +a2t +Untargeted Classification +Part-of-speech match, Max modification rate, SBERT sentence encoding cosine similarity, Word embedding distance +Counter-fitted word embedding swap (or BERT Masked Token Prediction, in the `mlm` variant) +Greedy-WIR (gradient) +Attack tuned for use in adversarial training, from "Towards Improving Adversarial Training of NLP Models" (Yoo et al., 2021) + alzantot Untargeted {Classification, Entailment} @@ -154,6 +162,14 @@ To run an attack recipe: `textattack attack --recipe [recipe_name]` Greedy attack with word importance ranking, reducing the input while maintaining the prediction through word importance ranking, from "Pathologies of Neural Models Make Interpretation Difficult" (Feng et al., 2018) +leap +Untargeted Classification +Max modification rate, Stopword modification +WordNet-based synonym swap +Particle Swarm Optimization (Levy-flight/adaptive-inertia variant) +Levy-flight/adaptive-inertia variant of the `pso` search method, from "LEAP: Efficient and Automated Test Method for NLP Software" (Ma et al., 2023) + + kuleshov Untargeted Classification Thought vector encoding cosine similarity, Language model similarity probability @@ -234,6 +250,33 @@ To run an attack recipe: `textattack attack --recipe [recipe_name]` Uses imperceptible character-level perturbations including homoglyph substitutions, Unicode reordering, deletions, and invisibles. Based on (["Bad Characters: Imperceptible NLP Attacks" (Boucher et al., 2021)](https://arxiv.org/abs/2106.09898)). +
Multi-lingual attacks on non-English classification models (Python API only, no CLI recipe name -- see #423):
 + + +FrenchRecipe +Untargeted Classification +Repeat modification, Stopword modification (French) +WordNet synonym swap, location swap, name swap (French) +Greedy-WIR +Contextualized-perturbation attack adapted for French NLP models. + + +SpanishRecipe +Untargeted Classification +Repeat modification, Stopword modification (Spanish) +WordNet synonym swap, location swap, name swap (Spanish) +Greedy-WIR +Contextualized-perturbation attack adapted for Spanish NLP models. + + +ChineseRecipe +Untargeted Classification +Repeat modification, Stopword modification (Chinese) +HowNet word swap, Masked-LM word swap, Morphonym & Homophone character swap +Greedy-WIR (weighted-saliency) +Contextualized-perturbation attack adapted for Chinese NLP models. + + diff --git a/docs/3recipes/models.md b/docs/3recipes/models.md index 0cd8bd67b..ad06146ac 100644 --- a/docs/3recipes/models.md +++ b/docs/3recipes/models.md @@ -393,9 +393,9 @@ All evaluations shown are on the full validation or test set up to 1000 examples ## How we have trained the TextAttack Models -- By Oct 2020, TextAttack provides users with 82 pre-trained TextAttack models, including word-level LSTM, word-level CNN, BERT, and other transformer based models pre-trained on various datasets provided by [HuggingFace](https://github.com/huggingface/nlp/). +- By Oct 2020, TextAttack provides users with 82 pre-trained TextAttack models, including word-level LSTM, word-level CNN, BERT, and other transformer based models pre-trained on various datasets provided by [HuggingFace](https://github.com/huggingface/datasets/). -- Since TextAttack is integrated with the [https://github.com/huggingface/nlp/](https://github.com/huggingface/nlp) library, it can automatically load the test or validation data set for the corresponding pre-trained model. While the literature has mainly focused on classification and entailment, TextAttack's pretrained models enable research on the robustness of models across all GLUE tasks. +- Since TextAttack is integrated with the [https://github.com/huggingface/datasets/](https://github.com/huggingface/datasets) library, it can automatically load the test or validation data set for the corresponding pre-trained model. While the literature has mainly focused on classification and entailment, TextAttack's pretrained models enable research on the robustness of models across all GLUE tasks. - We host all TextAttack Models at huggingface Model Hub: [https://huggingface.co/textattack](https://huggingface.co/textattack) From 00ef6c1aa24c8025cf0fc62a8342aac81a1d0495 Mon Sep 17 00:00:00 2001 From: Yanjun Qi / Jane Date: Fri, 14 Aug 2026 18:35:12 -0400 Subject: [PATCH 2/2] Explain why bad-characters is listed under "General" in recipe tables Unlike other recipes, which are locked to one task and one goal function, BadCharacters2021 takes a goal_function_type argument and can attack classification, NER, or seq2seq/translation models. Added a note in each of the three recipe tables/lists explaining why it doesn't fit the classification/seq2seq split used elsewhere. Co-Authored-By: Claude Sonnet 5 --- docs/1start/attacks4Components.md | 1 + docs/3recipes/attack_recipes.rst | 7 +++++++ docs/3recipes/attack_recipes_cmd.md | 1 + 3 files changed, 9 insertions(+) diff --git a/docs/1start/attacks4Components.md b/docs/1start/attacks4Components.md index 6c1275cd5..f187e65d4 100644 --- a/docs/1start/attacks4Components.md +++ b/docs/1start/attacks4Components.md @@ -229,6 +229,7 @@ A `SearchMethod` takes as input an initial `GoalFunctionResult` and returns a fi
General:
+Every other recipe here is locked to one task and one goal function, which is why the table splits into "classification" vs. "sequence-to-sequence" sections. bad-characters is different: its build() takes a goal_function_type argument and can be configured to attack classification models, NER models, or seq2seq/translation models. Since it spans both task types (and NER, which is neither), it doesn't fit either section above, so it's listed here under "General" instead. bad-characters diff --git a/docs/3recipes/attack_recipes.rst b/docs/3recipes/attack_recipes.rst index 6c0e5ce37..8f200acef 100644 --- a/docs/3recipes/attack_recipes.rst +++ b/docs/3recipes/attack_recipes.rst @@ -166,6 +166,13 @@ Attacks on sequence-to-sequence models General ############################################ +Unlike the recipes above, which are each locked to one task and one goal +function, ``BadCharacters2021`` takes a ``goal_function_type`` argument and +can be configured to attack classification models, NER models, or +seq2seq/translation models. Since it spans both of the sections above (and +NER, which is neither), it's listed here instead of being duplicated or +assigned to one arbitrarily. + 20. BadCharacters (Bad Characters: Imperceptible NLP Attacks) diff --git a/docs/3recipes/attack_recipes_cmd.md b/docs/3recipes/attack_recipes_cmd.md index 3118d26b8..bd0a8479b 100644 --- a/docs/3recipes/attack_recipes_cmd.md +++ b/docs/3recipes/attack_recipes_cmd.md @@ -240,6 +240,7 @@ To run an attack recipe: `textattack attack --recipe [recipe_name]`
General:
+Every other recipe here is locked to one task and one goal function, which is why the table splits into "classification" vs. "sequence-to-sequence" sections. bad-characters is different: its --recipe build takes a goal-function-type argument and can be configured to attack classification models, NER models, or seq2seq/translation models. Since it spans both task types (and NER, which is neither), it doesn't fit either section above, so it's listed here under "General" instead. bad-characters