Optimized skill for web tests generation#270
Conversation
GrimA9e
requested review from
uarlouski,
valfirst and
vkepin
and removed request for
a team
|
Warning Rate limit exceeded
Your organization is not enrolled in usage-based pricing. Contact your admin to enable usage-based pricing to continue reviews beyond the rate limit, or try again in 8 minutes and 40 seconds. ⌛ How to resolve this issue?After the wait time has elapsed, a review can be triggered using the We recommend that you space out your commits to avoid hitting the rate limit. 🚦 How do rate limits work?CodeRabbit enforces hourly rate limits for each developer per organization. Our paid plans have higher rate limits than the trial, open-source and free plans. In all cases, we re-allow further reviews after a brief timeout. Please see our FAQ for further information. ℹ️ Review info⚙️ Run configurationConfiguration used: Organization UI Review profile: ASSERTIVE Plan: Pro Run ID: 📒 Files selected for processing (1)
📝 WalkthroughWalkthroughWorkflow updated to generate only VIVIDUS Changes
Sequence Diagram(s)sequenceDiagram
participant User
participant Skill
participant VIVIDUS_MCP as "VIVIDUS MCP"
participant CONTEXT7_MCP as "context7 MCP"
participant Filesystem
User->>Skill: Provide input (steps, locators, context)
Skill->>VIVIDUS_MCP: Request available VIVIDUS steps
VIVIDUS_MCP-->>Skill: Return step list
alt Step syntax unclear
Skill->>CONTEXT7_MCP: Request context7 docs for clarification
CONTEXT7_MCP-->>Skill: Return docs
end
Skill->>Skill: Map input steps to available steps (or mark [MISSING STEP])
Skill->>Filesystem: Write single `.story` file (embed [ASSUMPTION] comments)
Filesystem-->>Skill: Ack
Skill-->>User: Deliver `.story` file
Estimated code review effort🎯 4 (Complex) | ⏱️ ~45 minutes Possibly related PRs
🚥 Pre-merge checks | ✅ 4 | ❌ 1❌ Failed checks (1 inconclusive)
✅ Passed checks (4 passed)
✏️ Tip: You can configure your own custom pre-merge checks in the settings. ✨ Finishing Touches🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 2
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.
Inline comments:
In @.github/skills/generate-vividus-web-tests/SKILL.md:
- Around line 59-61: The markdown has lint violations (MD058, MD022, MD031) due
to missing blank lines around tables, headings and fenced code blocks; fix by
inserting a blank line before and after the table that starts with "| Situation
| Assumption Made |", ensure each heading has an empty line above it and below
it, and add blank lines before and after the fenced code blocks (the example
using "gherkin" and the closing ``` fence); apply the same blank-line pattern to
the other occurrences referenced (the other table/heading/fence instances) so
all MD022/MD058/MD031 warnings are resolved.
- Around line 102-107: The policy text in SKILL.md currently contradicts itself
by saying "composite steps from `.steps` files take precedence" while rule 1
forbids using any steps not returned by VIVIDUS MCP; update the wording to
resolve this conflict by explicitly permitting project-level composite steps as
an allowed secondary source (in addition to MCP-returned steps) or by requiring
MCP discovery to include composites; modify the section containing "Priority
Rule" and the three "Strict rules" so it states that composite steps found in
project `.steps` files are valid and should be preferred when equivalent to MCP
steps, while still enforcing that any step not present in either MCP results or
project `.steps` must be marked `[MISSING STEP]` (refer to the "Priority Rule",
rule 1, and the guidance about `[MISSING STEP]`).
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Organization UI
Review profile: ASSERTIVE
Plan: Pro
Run ID: 84a46790-6432-40b2-82cc-62239967df61
📒 Files selected for processing (1)
.github/skills/generate-vividus-web-tests/SKILL.md
| 1. **Retrieve** test cases to automate | ||
| 2. **Execute** test cases with Playwright | ||
| 3. **Analyze** test case coverage | ||
| 3. **Analyze** test case and map to VIVIDUS |
There was a problem hiding this comment.
not clear: map to ... what?
| - `src/main/resources/story/**/*.story` — existing stories | ||
| - `src/main/resources/steps/*.steps` — reusable composite steps | ||
| 3. Lifecycle and Examples usage (transformers, data tables), scenario structure and naming, meta tags | ||
| 2. **MUST** obtain additional VIVIDUS documentation via **context7 MCP** when step syntax is unclear or additional capabilities need to be explored: |
There was a problem hiding this comment.
I think context7 should be optional
| - Call the MCP tool matching pattern `context7_query-docs` with the resolved library ID to look up specific step documentation, configuration options, or plugin capabilities | ||
| 3. **MUST** read existing resources to learn patterns and conventions: | ||
| - `src/main/resources/story/**/*.story` — existing stories | ||
| - `src/main/resources/steps/*.steps` — reusable composite steps |
There was a problem hiding this comment.
I think the paths should be moved to instructions
| 1. **Locators:** Follow the **Locator Stability Hierarchy** below to ensure stability. | ||
| 2. **Expressions:** Use VIVIDUS expressions instead of hardcoded dynamic values — see **Use VIVIDUS Expressions Instead of Hardcoded Values** below. | ||
| 3. **Data Tables:** Use Examples blocks **only** when a scenario must run with multiple distinct data sets. Do NOT use Examples for a single data set — inline values or expressions directly. | ||
| 4. **Composite Steps:** Propose new composite steps for repeated action patterns. |
There was a problem hiding this comment.
| 4. **Composite Steps:** Propose new composite steps for repeated action patterns. | |
| 4. **Composite Steps:** Create new composite steps for repeated action patterns. |
?
| - **Generate date**: `#{generateDate($period, $outputFormat)}` — e.g. `#{generateDate(P, yyyy-MM-dd)}` (today), `#{generateDate(P1D, yyyy-MM-dd)}` (tomorrow) | ||
| - **Fake data** (via DataFaker): `#{generate($Provider.$method)}` — e.g. `#{generate(Name.firstName)}`, `#{generate(Internet.emailAddress)}`, `#{generate(Address.fullAddress)}` | ||
| - **Pick random from list**: `#{anyOf($value1, $value2, $value3)}` | ||
| - **String transforms**: `#{toLowerCase($input)}`, `#{toUpperCase($input)}` |
There was a problem hiding this comment.
I think it should be moved to MCP server
| #### File 1: VIVIDUS Story | ||
| **Location**: `src/main/resources/story/generated/TC-XXXXX-[TestName]/[TestName].story` | ||
|
|
||
| **DO NOT create** any additional files (README, summary reports, documentation, etc.) — only the `.story` file. |
There was a problem hiding this comment.
what about composite steps?
| @requirementId [Requirement Id] | ||
| @feature [Feature] | ||
| @priority [0|1|2|3|4] | ||
| @priority [1|2|3|4] |
There was a problem hiding this comment.
| @priority [1|2|3|4] | |
| @priority [1|2|3|4|5] |
|
|
||
| !-- [MISSING STEP] Comment for any gaps | ||
| !-- [ASSUMPTION] Comment for any assumptions made - REQUIRES VALIDATION | ||
| !-- [ASSUMPTION] Comment for any assumptions made — REQUIRES VALIDATION |
| | `@requirementId` | `REQ-XXXXX` | Linked requirement/user story ID | | ||
| | `@feature` | Feature name | Should match feature folder or test suite | | ||
| | `@priority` | `1` \| `2` \| `3` \| `4` \| `5` | 1=Blocker, 2=Critical, 3=Major, 4=Minor, 5=Trivial | | ||
| | `@priority` | `1` \| `2` \| `3` \| `4` | 1=Critical, 2=Major, 3=Minor, 4=Trivial | |
| **When to use visual testing**: | ||
| - ✅ Verifying page layout, structure, elements and their states (3+ elements) | ||
| - ❌ Single element verification after an action | ||
| - ❌ Dynamic content that changes frequently |
There was a problem hiding this comment.
should it be extracted to another skill?
GrimA9e
force-pushed
the
web_skill_update
branch
from
89f2c9b to
c0c723d
Compare
There was a problem hiding this comment.
Actionable comments posted: 11
Caution
Some comments are outside the diff and can’t be posted inline due to platform limitations.
⚠️ Outside diff range comments (2)
.github/skills/generate-vividus-web-tests/SKILL.md (2)
221-229:⚠️ Potential issue | 🟡 MinorAdd blank line before code block.
A blank line is needed before the code fence (line 223) to satisfy markdownlint rule MD031 (blanks-around-fences).
📝 Suggested fix
❌ **Bad** - waiting before every field (unnecessary): ```gherkin🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In @.github/skills/generate-vividus-web-tests/SKILL.md around lines 221 - 229, The Markdown in SKILL.md has a code fence immediately after text which violates MD031; insert a single blank line before the opening code fence (the ```gherkin block shown in the example) so there is an empty line between the preceding paragraph and the code block, ensuring the Gherkin snippet is separated and MD031 passes.
203-211:⚠️ Potential issue | 🟡 MinorAdd blank line before code block.
A blank line is needed before the code fence (line 204) to satisfy markdownlint rule MD031 (blanks-around-fences).
📝 Suggested fix
✅ **Good** - wait for first interactive element after navigation: ```gherkin🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In @.github/skills/generate-vividus-web-tests/SKILL.md around lines 203 - 211, Add a blank line immediately before the opening Gherkin code fence (the sequence ```gherkin) in SKILL.md so the fence is separated by a blank line from the preceding paragraph; locate the triple-backtick fence and insert one empty line above it to satisfy markdownlint MD031.
♻️ Duplicate comments (7)
.github/skills/generate-vividus-web-tests/SKILL.md (7)
59-65:⚠️ Potential issue | 🟡 MinorAdd blank line before table to fix markdownlint violation.
The table starting at line 60 needs a blank line before it to satisfy markdownlint rule MD058 (blanks-around-tables).
This issue was flagged in a previous review and remains unresolved.
📝 Suggested fix
3. Flag with an `[ASSUMPTION]` inline comment in the story file Example assumptions: + | Situation | Assumption Made |🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In @.github/skills/generate-vividus-web-tests/SKILL.md around lines 59 - 65, Add a blank line immediately before the Markdown table that begins with the header "| Situation | Assumption Made |" to satisfy markdownlint rule MD058 (blanks-around-tables); update SKILL.md by inserting an empty line above that table so the table is separated from the preceding paragraph/content.
133-139:⚠️ Potential issue | 🟡 MinorAdd blank line before code block.
A blank line is needed before the code fence (line 135) to satisfy markdownlint rule MD031 (blanks-around-fences). This was flagged in a previous review.
📝 Suggested fix
❌ **Bad** - hardcoded values: ```gherkin🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In @.github/skills/generate-vividus-web-tests/SKILL.md around lines 133 - 139, Add a blank line immediately before the opening code fence that contains the Gherkin example (the triple backticks before the lines starting with "When I enter `John`..." and the other two "When" lines) so the code block is preceded by an empty line to satisfy markdownlint rule MD031; ensure the fence remains labeled (e.g., ```gherkin) and that only a single empty line is inserted directly above the fence.
97-99: 🧹 Nitpick | 🔵 TrivialConsider making context7 MCP optional.
The current wording mandates context7 MCP ("MUST obtain additional VIVIDUS documentation"), but a previous reviewer suggested it should be optional. Since VIVIDUS MCP (line 94) is the primary source and context7 is supplementary for unclear syntax, consider changing this to "SHOULD" or "MAY" to allow agents to proceed when context7 is unavailable.
As per coding guidelines, addressing reviewer feedback: "I think
context7should be optional"♻️ Suggested wording adjustment
-2. **MUST** obtain additional VIVIDUS documentation via **context7 MCP** when step syntax is unclear or additional capabilities need to be explored: +2. **SHOULD** obtain additional VIVIDUS documentation via **context7 MCP** when step syntax is unclear or additional capabilities need to be explored (optional):🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In @.github/skills/generate-vividus-web-tests/SKILL.md around lines 97 - 99, The requirement in the SKILL.md clause that reads "MUST obtain additional VIVIDUS documentation via context7 MCP" should be relaxed to make context7 optional; update the phrase to use "SHOULD" or "MAY" (e.g., replace "MUST" with "SHOULD" or "MAY") in the sentence that references "context7 MCP" and the two subsequent bullet actions ("context7_resolve-library-id" and "context7_query-docs"), so agents may proceed using VIVIDUS MCP as the primary source when context7 is unavailable while still recommending context7 when step syntax is unclear.
85-85:⚠️ Potential issue | 🟡 MinorAdd blank line before subheading.
The "Logic & Flow Planning" subheading needs a blank line before it to satisfy markdownlint rule MD022 (blanks-around-headings).
📝 Suggested fix
## Step 3: Analyze test case and map to VIVIDUS ### Logic & Flow Planning🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In @.github/skills/generate-vividus-web-tests/SKILL.md at line 85, Add a single blank line immediately before the "Logic & Flow Planning" subheading in the SKILL.md content so the markdown heading has a blank line above it (to satisfy markdownlint MD022); locate the heading text "Logic & Flow Planning" and insert one empty line directly above that line.
241-241:⚠️ Potential issue | 🟡 MinorAdd blank line before subheading.
The "Output" subheading needs a blank line before it to satisfy markdownlint rule MD022 (blanks-around-headings).
📝 Suggested fix
## Step 5: Generate VIVIDUS story ### Output🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In @.github/skills/generate-vividus-web-tests/SKILL.md at line 241, Add a blank line immediately before the "### Output" subheading (the line containing "### Output") so the markdown has an empty line above that heading to satisfy MD022 (blanks-around-headings).
91-91:⚠️ Potential issue | 🟡 MinorAdd blank line before subheading.
The "Discovery" subheading needs a blank line before it to satisfy markdownlint rule MD022 (blanks-around-headings).
📝 Suggested fix
3. Check for specific negative scenarios: Ensure the test verifies the *fail state* (e.g., "Error message visible") and doesn't accidentally succeed if the error is missing. ### Discovery🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In @.github/skills/generate-vividus-web-tests/SKILL.md at line 91, Add a single blank line immediately before the "### Discovery" subheading in the SKILL.md content so the heading is separated from the previous paragraph and satisfies markdownlint MD022; locate the literal heading "### Discovery" and insert one blank line above it.
149-149:⚠️ Potential issue | 🟡 MinorAdd blank line before subheading.
The "Locator Stability Hierarchy" subheading needs a blank line before it to satisfy markdownlint rule MD022 (blanks-around-headings).
📝 Suggested fix
Locator Stability Hierarchy
</details> <details> <summary>🤖 Prompt for AI Agents</summary>Verify each finding against the current code and only fix it if needed.
In @.github/skills/generate-vividus-web-tests/SKILL.md at line 149, Add a single
blank line immediately before the "### Locator Stability Hierarchy" heading to
satisfy markdownlint MD022; locate the heading text "### Locator Stability
Hierarchy" in SKILL.md and insert one empty line above it so the subheading is
separated from the preceding paragraph or element.</details> </blockquote></details> </blockquote></details> <details> <summary>🤖 Prompt for all review comments with AI agents</summary>Verify each finding against the current code and only fix it if needed.
Inline comments:
In @.github/skills/generate-vividus-web-tests/SKILL.md:
- Line 119: Update the guidance text under the "Composite Steps" bullet so it is
directive: replace the sentence "Propose new composite steps for repeated action
patterns." with "Create new composite steps for repeated action
patterns."—locate the "Composite Steps" bullet in SKILL.md (the line currently
reading the less-direct wording) and make the wording change so it reads as an
expected action rather than a suggestion.- Line 247: Update the SKILL.md guidance to explicitly state that agents must
not create any new files including composite step files (e.g., no new.steps
files) and instead, if proposing reusable or composite steps, they should
include them inline as clearly commented blocks within the single.storyfile
(e.g., use a "COMPOSITE STEP:" comment header inside the.story) so reviewers
can extract or refactor them later; reference the existing rule "DO NOT create
any additional files — only the.storyfile" and add this clarification
immediately after it to remove ambiguity about composite steps.- Around line 100-103: Move the two specific resource paths currently listed
under the Discovery section (src/main/resources/story/**/*.storyand
src/main/resources/steps/*.steps) into the Instructions section of SKILL.md so
Discovery remains tool-focused; remove those lines from the Discovery bullets
and add a short "Project conventions" or "Resource paths" subsection under
Instructions that lists those paths and a brief note about their purpose (e.g.,
"existing stories" and "reusable composite steps"), and ensure the
numbering/formatting in both sections is updated accordingly.- Line 11: The phrase "3. Analyze test case and map to VIVIDUS" is
ambiguous; update that line to explicitly state the mapping target and action —
e.g., change it to "3. Analyze test case and translate to VIVIDUS steps and
syntax" or "3. Analyze test case and map to VIVIDUS steps/syntax" so it's
clear whether you're converting to VIVIDUS steps, syntax, or capabilities;
locate and edit the exact text "3. Analyze test case and map to VIVIDUS" in
SKILL.md.- Around line 105-110: Update the Priority Rule wording to explicitly state
operational precedence: change the sentence that currently says "Two sources of
valid steps are allowed... composite steps defined in project.stepsfiles —
these take precedence, and (2) steps returned by VIVIDUS MCP" to a clear
directive such as "When both a composite step from project.stepsfiles and a
step returned by VIVIDUS MCP can accomplish the same action, always prefer and
use the composite step from the project.stepsfile; only use the MCP step if
no equivalent composite exists." Ensure this replaces the existing priority
sentence near the "Priority Rule" header and keep the surrounding strict rules
(ONLY use steps from project.stepsfiles OR VIVIDUS MCP results, Preserve
exact syntax, mark[MISSING STEP]) intact.- Line 122: Insert a single blank line immediately before the subheading "###
Use VIVIDUS Expressions Instead of Hardcoded Values" in SKILL.md so the heading
is separated from the preceding content, satisfying markdownlint MD022; locate
the exact heading text and add one empty line above it.- Around line 164-168: Add a blank line immediately before the Gherkin code
fence so the snippet beginning with "gherkin" is preceded by an empty line to satisfy markdownlint MD031; update the SKILL.md section containing the Gherkin example (the block that starts with "When I wait until element located by `caseInsensitiveText(My Account)` appears") by inserting one blank line above the openinggherkin fence.- Around line 141-147: Add a blank line directly before the Gherkin code fence
under the "✅ Good - generated values:" block so the triple-backtick fence
that starts the code block is preceded by an empty line; update the SKILL.md
section containing the sample Gherkin snippet (the block that begins with "When
I enter#{generate(Name.firstName)}...") by inserting one empty line before
the opening ```gherkin fence to satisfy MD031.- Around line 213-219: Add a blank line immediately before the Gherkin code
fence that begins with "gherkin" in SKILL.md so the code block is separated from the preceding paragraph (this satisfies markdownlint MD031); locate the fence shown with the snippet starting "gherkin" and insert a single empty
line above it.- Around line 273-280: Add a blank line immediately before the code fence that
starts with ```gherkin (the block containing the two Gherkin steps "When I click
on element located bybuttonName(Save)" and "When I wait until element located
by `caseInsensitiveText(Success)` appears in `PT10S`") so the fence is separated
by an empty line to satisfy markdownlint MD031.- Line 254: Update the
@priorityenumeration and its explanatory table to
include level 5: change the tag instance@priority [1|2|3|4]to@priority [1|2|3|4|5]and update the table row that documents@priorityto list1 | 2 | 3 | 4 | 5with the corresponding descriptions (add5=Informational) so the
docs reflect VIVIDUS's five priority levels.
Outside diff comments:
In @.github/skills/generate-vividus-web-tests/SKILL.md:
- Around line 221-229: The Markdown in SKILL.md has a code fence immediately
after text which violates MD031; insert a single blank line before the opening
code fence (the ```gherkin block shown in the example) so there is an empty line
between the preceding paragraph and the code block, ensuring the Gherkin snippet
is separated and MD031 passes.- Around line 203-211: Add a blank line immediately before the opening Gherkin
code fence (the sequence ```gherkin) in SKILL.md so the fence is separated by a
blank line from the preceding paragraph; locate the triple-backtick fence and
insert one empty line above it to satisfy markdownlint MD031.
Duplicate comments:
In @.github/skills/generate-vividus-web-tests/SKILL.md:
- Around line 59-65: Add a blank line immediately before the Markdown table that
begins with the header "| Situation | Assumption Made |" to satisfy markdownlint
rule MD058 (blanks-around-tables); update SKILL.md by inserting an empty line
above that table so the table is separated from the preceding paragraph/content.- Around line 133-139: Add a blank line immediately before the opening code
fence that contains the Gherkin example (the triple backticks before the lines
starting with "When I enterJohn..." and the other two "When" lines) so the
code block is preceded by an empty line to satisfy markdownlint rule MD031;
ensure the fence remains labeled (e.g., ```gherkin) and that only a single empty
line is inserted directly above the fence.- Around line 97-99: The requirement in the SKILL.md clause that reads "MUST
obtain additional VIVIDUS documentation via context7 MCP" should be relaxed to
make context7 optional; update the phrase to use "SHOULD" or "MAY" (e.g.,
replace "MUST" with "SHOULD" or "MAY") in the sentence that references "context7
MCP" and the two subsequent bullet actions ("context7_resolve-library-id" and
"context7_query-docs"), so agents may proceed using VIVIDUS MCP as the primary
source when context7 is unavailable while still recommending context7 when step
syntax is unclear.- Line 85: Add a single blank line immediately before the "Logic & Flow
Planning" subheading in the SKILL.md content so the markdown heading has a blank
line above it (to satisfy markdownlint MD022); locate the heading text "Logic &
Flow Planning" and insert one empty line directly above that line.- Line 241: Add a blank line immediately before the "### Output" subheading (the
line containing "### Output") so the markdown has an empty line above that
heading to satisfy MD022 (blanks-around-headings).- Line 91: Add a single blank line immediately before the "### Discovery"
subheading in the SKILL.md content so the heading is separated from the previous
paragraph and satisfies markdownlint MD022; locate the literal heading "###
Discovery" and insert one blank line above it.- Line 149: Add a single blank line immediately before the "### Locator
Stability Hierarchy" heading to satisfy markdownlint MD022; locate the heading
text "### Locator Stability Hierarchy" in SKILL.md and insert one empty line
above it so the subheading is separated from the preceding paragraph or element.</details> <details> <summary>🪄 Autofix (Beta)</summary> Fix all unresolved CodeRabbit comments on this PR: - [ ] <!-- {"checkboxId": "4b0d0e0a-96d7-4f10-b296-3a18ea78f0b9"} --> Push a commit to this branch (recommended) - [ ] <!-- {"checkboxId": "ff5b1114-7d8c-49e6-8ac1-43f82af23a33"} --> Create a new PR with the fixes </details> --- <details> <summary>ℹ️ Review info</summary> <details> <summary>⚙️ Run configuration</summary> **Configuration used**: Organization UI **Review profile**: ASSERTIVE **Plan**: Pro **Run ID**: `c67af08a-0e64-41cf-954d-ff91bf57891f` </details> <details> <summary>📥 Commits</summary> Reviewing files that changed from the base of the PR and between 89f2c9b14b305c02be1927eb148985e6600d3bfb and c0c723d64489cec55ea7d7e36d9152128fdbd4aa. </details> <details> <summary>📒 Files selected for processing (1)</summary> * `.github/skills/generate-vividus-web-tests/SKILL.md` </details> </details> <!-- This is an auto-generated comment by CodeRabbit for review status -->
| 3. **MUST** read existing resources to learn patterns and conventions: | ||
| - `src/main/resources/story/**/*.story` — existing stories | ||
| - `src/main/resources/steps/*.steps` — reusable composite steps | ||
| 4. **MUST** review Lifecycle and Examples usage (transformers, data tables), scenario structure and naming, meta tags |
There was a problem hiding this comment.
🧹 Nitpick | 🔵 Trivial
Consider moving file paths to instructions section.
A previous reviewer suggested moving the specific file paths (src/main/resources/story/**/*.story, src/main/resources/steps/*.steps) from the Discovery section to a separate instructions section for better organization. This would keep Discovery focused on tool usage while instructions detail project-specific patterns.
As per coding guidelines, addressing reviewer feedback: "I think the paths should be moved to instructions"
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.
In @.github/skills/generate-vividus-web-tests/SKILL.md around lines 100 - 103,
Move the two specific resource paths currently listed under the Discovery
section (`src/main/resources/story/**/*.story` and
`src/main/resources/steps/*.steps`) into the Instructions section of SKILL.md so
Discovery remains tool-focused; remove those lines from the Discovery bullets
and add a short "Project conventions" or "Resource paths" subsection under
Instructions that lists those paths and a brief note about their purpose (e.g.,
"existing stories" and "reusable composite steps"), and ensure the
numbering/formatting in both sections is updated accordingly.
| 1. **Locators:** Follow the **Locator Stability Hierarchy** below to ensure stability. | ||
| 2. **Expressions:** Use VIVIDUS expressions instead of hardcoded dynamic values — see **Use VIVIDUS Expressions Instead of Hardcoded Values** below. | ||
| 3. **Data Tables:** Use Examples blocks **only** when a scenario must run with multiple distinct data sets. Do NOT use Examples for a single data set — inline values or expressions directly. | ||
| 4. **Composite Steps:** Propose new composite steps for repeated action patterns. |
There was a problem hiding this comment.
🧹 Nitpick | 🔵 Trivial
Consider rewording composite steps guidance.
A previous reviewer suggested alternative wording: "Composite Steps: Create new composite steps for repeated action patterns."
The current wording "Propose new composite steps" is less directive. Consider the suggested wording to make it clearer that creating composites is an expected action, not just a suggestion.
As per coding guidelines, addressing reviewer feedback at line 119.
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.
In @.github/skills/generate-vividus-web-tests/SKILL.md at line 119, Update the
guidance text under the "Composite Steps" bullet so it is directive: replace the
sentence "Propose new composite steps for repeated action patterns." with
"Create new composite steps for repeated action patterns."—locate the "Composite
Steps" bullet in SKILL.md (the line currently reading the less-direct wording)
and make the wording change so it reads as an expected action rather than a
suggestion.
| 4. **Composite Steps:** Propose new composite steps for repeated action patterns. | ||
| 5. **Contextual Steps:** When using parent element context, ensure child locators are relative. | ||
|
|
||
| ### Use VIVIDUS Expressions Instead of Hardcoded Values |
There was a problem hiding this comment.
Add blank line before subheading.
The "Use VIVIDUS Expressions Instead of Hardcoded Values" subheading needs a blank line before it to satisfy markdownlint rule MD022 (blanks-around-headings).
📝 Suggested fix
5. **Contextual Steps:** When using parent element context, ensure child locators are relative.
### Use VIVIDUS Expressions Instead of Hardcoded Values📝 Committable suggestion
‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.
| ### Use VIVIDUS Expressions Instead of Hardcoded Values | |
| 5. **Contextual Steps:** When using parent element context, ensure child locators are relative. | |
| ### Use VIVIDUS Expressions Instead of Hardcoded Values |
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.
In @.github/skills/generate-vividus-web-tests/SKILL.md at line 122, Insert a
single blank line immediately before the subheading "### Use VIVIDUS Expressions
Instead of Hardcoded Values" in SKILL.md so the heading is separated from the
preceding content, satisfying markdownlint MD022; locate the exact heading text
and add one empty line above it.
|
|
||
| ```gherkin | ||
| When I wait until element located by `caseInsensitiveText(My Account)` appears | ||
| Then text `My Account` exists | ||
| ``` |
There was a problem hiding this comment.
Add blank line before code block.
A blank line is needed before the code fence (line 165) to satisfy markdownlint rule MD031 (blanks-around-fences).
📝 Suggested fix
❌ **Bad** - redundant check:
```gherkin🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.
In @.github/skills/generate-vividus-web-tests/SKILL.md around lines 164 - 168,
Add a blank line immediately before the Gherkin code fence so the snippet
beginning with "```gherkin" is preceded by an empty line to satisfy markdownlint
MD031; update the SKILL.md section containing the Gherkin example (the block
that starts with "When I wait until element located by `caseInsensitiveText(My
Account)` appears") by inserting one blank line above the opening ```gherkin
fence.
| ❌ **Bad** - no synchronization after navigation: | ||
|
|
||
| ```gherkin | ||
| When I click on element located by `buttonName(Add Product)` | ||
| !-- Missing wait - next step may fail if page hasn't loaded | ||
| When I enter `${campaignName}` in field located by `xpath(//input[@name='name'])` | ||
| ``` |
There was a problem hiding this comment.
Add blank line before code block.
A blank line is needed before the code fence (line 215) to satisfy markdownlint rule MD031 (blanks-around-fences).
📝 Suggested fix
❌ **Bad** - no synchronization after navigation:
```gherkin🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.
In @.github/skills/generate-vividus-web-tests/SKILL.md around lines 213 - 219,
Add a blank line immediately before the Gherkin code fence that begins with
"```gherkin" in SKILL.md so the code block is separated from the preceding
paragraph (this satisfies markdownlint MD031); locate the fence shown with the
snippet starting "```gherkin" and insert a single empty line above it.
| @requirementId [Requirement Id] | ||
| @feature [Feature] | ||
| @priority [0|1|2|3|4] | ||
| @priority [1|2|3|4] |
There was a problem hiding this comment.
🧩 Analysis chain
🏁 Script executed:
#!/bin/bash
# Description: Find all priority values used in existing VIVIDUS stories
# Search for `@priority` tags in story files and extract the values
rg -oN '@priority\s+\d+' --glob '*.story' | sort -uRepository: vividus-framework/vividus-sample-tests
Length of output: 64
🏁 Script executed:
cat -n .github/skills/generate-vividus-web-tests/SKILL.md | sed -n '250,275p'Repository: vividus-framework/vividus-sample-tests
Length of output: 1112
🏁 Script executed:
rg -i 'priority' .github/skills/generate-vividus-web-tests/SKILL.md -A 2 -B 2Repository: vividus-framework/vividus-sample-tests
Length of output: 919
🏁 Script executed:
fd -type f -name '*.md' | xargs rg -l 'priority.*level' -iRepository: vividus-framework/vividus-sample-tests
Length of output: 255
🏁 Script executed:
rg -i 'priority.*[5-9]|level.*5|highest|lowest' . -t md -t txt --max-count 20Repository: vividus-framework/vividus-sample-tests
Length of output: 64
🌐 Web query:
VIVIDUS framework priority levels @priority meta tag documentation
💡 Result:
The VIVIDUS framework uses the @priority meta tag to assign priority levels to stories and scenarios for reporting purposes, specifically to view the distribution of tests by priorities in the generated report's Graphs tab. Priority levels are numeric values in the range 1-5, where typically the smallest number (1) indicates the most important test, and the highest number (5) indicates the least important one. Usage example: Meta: @priority 1 Note: The older @severity meta tag served the same purpose but is deprecated and scheduled for removal in VIVIDUS 0.7.0; users should migrate to @priority. This meta tag can be placed at the story or scenario level. After running tests, the priority distribution appears in the report under the Graphs tab. There is a separate mechanism for story execution prioritization using a different meta tag like @story_priority, configurable via Spring bean, but the queried @priority is specifically for reporting.
Citations:
- 1: https://docs.vividus.dev/vividus/0.6.5/reporting.html
- 2: https://docs.vividus.dev/vividus/latest/configuration/reporting.html
- 3: https://docs.vividus.dev/vividus/0.5.2/tests-configuration.html
🌐 Web query:
VIVIDUS @priority 5 definition name "trivial" "informational"
💡 Result:
In VIVIDUS test automation framework, @priority 5 with names "trivial" and "informational" likely refers to the priority level configuration for known issues (configured in known-issues.json). Priority 5 is the lowest level, corresponding to "trivial" issues that are purely informational and do not impact test execution significantly. Known issues at this level are matched by assertion patterns and treated as non-failing by default unless configured otherwise (via CLI flags like --treatKnownIssuesAsPassed or build.gradle properties). Examples from docs show known issues labeled like VVD-5 with low priority patterns for minor assertions.
Citations:
- 1: https://docs.vividus.dev/vividus/0.5.5/commons/cli.html
- 2: https://docs.vividus.dev/vividus/latest/commons/cli.html
🌐 Web query:
VIVIDUS @priority meta tag levels 1 2 3 4 5 critical major minor trivial informational
💡 Result:
In VIVIDUS, the @priority meta tag is used at the story or scenario level to mark the importance of tests for reporting purposes, specifically to view the distribution of tests by priorities in the Graphs tab of the report. @severity is deprecated in favor of @priority since VIVIDUS 0.7.0. Allowed values: Numeric values in the range 1-5, where typically the smallest number (1) indicates the most important test, and the highest number (5) indicates the least important one. Examples: - @priority 1 (highest importance) - @priority 2 - @priority 3 - @priority 4 - @priority 5 (lowest importance) No official mapping to qualitative labels like critical, major, minor, trivial, informational is documented in the sources. These terms appear in general software testing contexts but are not explicitly linked to VIVIDUS priority levels. Users may conventionally map them as: - 1: critical - 2: major - 3: minor - 4: trivial - 5: informational A separate story_priority meta can be used for execution order prioritization (higher number starts first), but that's distinct from @priority for reporting.
Citations:
- 1: https://docs.vividus.dev/vividus/0.6.5/reporting.html
- 2: https://docs.vividus.dev/vividus/latest/configuration/reporting.html
- 3: https://www.programmersought.com/article/9730446258/
- 4: https://www.cobeisfresh.com/blog/prioritisation-in-quality-assurance-how-we-do-it
Update priority levels to include level 5.
The priority tag currently shows only 4 levels [1|2|3|4], but VIVIDUS supports 5 priority levels (1–5) for test distribution reporting. Line 254 and line 270 both need updating:
- Line 254: Change
@priority [1|2|3|4]to@priority [1|2|3|4|5] - Line 270: Add level 5 to the table:
|@priority|1\|2\|3\|4\|5| 1=Critical, 2=Major, 3=Minor, 4=Trivial, 5=Informational |
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.
In @.github/skills/generate-vividus-web-tests/SKILL.md at line 254, Update the
`@priority` enumeration and its explanatory table to include level 5: change the
tag instance `@priority [1|2|3|4]` to `@priority [1|2|3|4|5]` and update the
table row that documents `@priority` to list `1 | 2 | 3 | 4 | 5` with the
corresponding descriptions (add `5=Informational`) so the docs reflect VIVIDUS's
five priority levels.
|
|
||
| ```gherkin | ||
| !-- [ASSUMPTION] TC step said "click submit" but button text is "Save" - using "Save" | ||
| When I click on element located by `xpath(//button[text()='Save'])` | ||
| !-- [ASSUMPTION] TC step said "click submit" but button text is "Save" — using "Save" | ||
| When I click on element located by `buttonName(Save)` | ||
|
|
||
| !-- [ASSUMPTION] TC doesn't specify wait time, assuming 10 seconds max | ||
| When I wait until element located by `caseInsensitiveText(Success)` appears in `PT10S` | ||
| ``` |
There was a problem hiding this comment.
Add blank line before code block.
A blank line is needed before the code fence (line 274) to satisfy markdownlint rule MD031 (blanks-around-fences).
📝 Suggested fix
**Assumption Comments in Story:**
```gherkin🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.
In @.github/skills/generate-vividus-web-tests/SKILL.md around lines 273 - 280,
Add a blank line immediately before the code fence that starts with ```gherkin
(the block containing the two Gherkin steps "When I click on element located by
`buttonName(Save)`" and "When I wait until element located by
`caseInsensitiveText(Success)` appears in `PT10S`") so the fence is separated by
an empty line to satisfy markdownlint MD031.
c0c723d to
7177155
Compare
2 participants
Summary by CodeRabbit