-
Notifications
You must be signed in to change notification settings - Fork 115
chore(e2e): scorecard grouped metric tests #4196
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Merged
jrichter1
merged 15 commits into
redhat-developer:main
from
teknaS47:scorecard-grouped-metrics-e2e
Aug 11, 2026
Merged
Changes from all commits
Commits
Show all changes
15 commits
Select commit
Hold shift + click to select a range
c56d2bc
scorecard grouped metric tests
teknaS47 281ae3d
chore(deps): update dependency @playwright/test to v1.62.1 (#4192)
renovate[bot] c48f025
chore(deps): update yarn to v4.17.1 (#2918)
renovate[bot] 493364b
fix(deps): update dependency @openai/agents-core to ^0.14.0 (#4118)
renovate[bot] ca9c77e
fullsend suggested changes
teknaS47 1ec7996
Merge branch 'main' into scorecard-grouped-metrics-e2e
teknaS47 e144d4e
ci: retrigger with updated base
teknaS47 5fdd3ef
tsconfig change
teknaS47 29dc346
Merge branch 'main' into scorecard-grouped-metrics-e2e
teknaS47 3e28726
translation fix
teknaS47 0a7a98c
Merge branch 'main' into scorecard-grouped-metrics-e2e
teknaS47 29704e1
adressing comments from Jan
teknaS47 57589d6
adressing comments
teknaS47 ad5d150
Merge branch 'main' into scorecard-grouped-metrics-e2e
teknaS47 9b5ad50
Merge branch 'main' into scorecard-grouped-metrics-e2e
teknaS47 File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
238 changes: 238 additions & 0 deletions
238
workspaces/scorecard/packages/app-legacy/e2e-tests/scorecard-nfs.test.ts
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,238 @@ | ||
| /* | ||
| * Copyright Red Hat, Inc. | ||
| * | ||
| * Licensed under the Apache License, Version 2.0 (the "License"); | ||
| * you may not use this file except in compliance with the License. | ||
| * You may obtain a copy of the License at | ||
| * | ||
| * http://www.apache.org/licenses/LICENSE-2.0 | ||
| * | ||
| * Unless required by applicable law or agreed to in writing, software | ||
| * distributed under the License is distributed on an "AS IS" BASIS, | ||
| * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. | ||
| * See the License for the specific language governing permissions and | ||
| * limitations under the License. | ||
| */ | ||
|
|
||
| import { test, expect, Page } from '@playwright/test'; | ||
| import { mockSonarqubeScorecardResponse } from './utils/apiUtils'; | ||
| import { CatalogPage } from './pages/CatalogPage'; | ||
| import { ScorecardPage } from './pages/ScorecardPage'; | ||
| import { getTranslations, ScorecardMessages } from './utils/translationUtils'; | ||
| import { | ||
| sonarqubeScorecardResponse, | ||
| sonarqubeFailedQualityGateResponse, | ||
| } from './utils/scorecardResponseUtils'; | ||
| import { runAccessibilityTests } from './utils/accessibility'; | ||
| import { installWebpackDevOverlayGuards } from './utils/devOverlays'; | ||
|
|
||
| test.describe('Metric Group Cards', () => { | ||
|
jrichter1 marked this conversation as resolved.
|
||
| let page: Page; | ||
| let catalogPage: CatalogPage; | ||
| let scorecardPage: ScorecardPage; | ||
| let translations: ScorecardMessages; | ||
|
|
||
| test.beforeAll(async ({ browser }, testInfo) => { | ||
| const locale = (testInfo.project.use.locale as string | undefined) ?? 'en'; | ||
| const context = await browser.newContext({ locale }); | ||
| await context.addInitScript(installWebpackDevOverlayGuards); | ||
| page = await context.newPage(); | ||
| const currentLocale = await page.evaluate( | ||
|
jrichter1 marked this conversation as resolved.
|
||
| () => globalThis.navigator.language, | ||
| ); | ||
| translations = getTranslations(currentLocale); | ||
|
|
||
| catalogPage = new CatalogPage(page); | ||
| scorecardPage = new ScorecardPage(page, translations); | ||
|
|
||
| await catalogPage.loginAndSetLocale(currentLocale); | ||
|
|
||
| await mockSonarqubeScorecardResponse(page, sonarqubeScorecardResponse); | ||
| await catalogPage.openCatalog(); | ||
| await catalogPage.openComponent('sonarqube-scorecard-only'); | ||
| await page.getByText('Scorecard', { exact: true }).click(); | ||
| await expect(page.getByLabel('Security Vulnerabilities')).toBeVisible({ | ||
| timeout: 15000, | ||
| }); | ||
| }); | ||
|
|
||
| test.afterAll(async () => { | ||
| await page?.context()?.close(); | ||
| }); | ||
|
|
||
| test('Verify group cards render with titles, descriptions, and bucket tiles', async ({}, testInfo) => { | ||
| const securityCard = scorecardPage.getGroupCard('Security Vulnerabilities'); | ||
| await expect(page.getByLabel('Security Vulnerabilities')).toBeVisible(); | ||
| await expect( | ||
| securityCard.getByText('Track security issues across your repositories'), | ||
| ).toBeVisible(); | ||
| await expect( | ||
| scorecardPage.getBucketTile(securityCard, 'success'), | ||
|
teknaS47 marked this conversation as resolved.
|
||
| ).toContainText('1'); | ||
|
|
||
| const codeQualityCard = scorecardPage.getGroupCard('Code Quality'); | ||
| await expect(page.getByLabel('Code Quality')).toBeVisible(); | ||
| await expect( | ||
| codeQualityCard.getByText('Code quality and maintainability metrics'), | ||
| ).toBeVisible(); | ||
| await expect( | ||
| scorecardPage.getBucketTile(codeQualityCard, 'success'), | ||
| ).toContainText('3'); | ||
|
|
||
| const coverageCard = scorecardPage.getGroupCard('SonarQube Coverage'); | ||
| await expect(page.getByLabel('SonarQube Coverage')).toBeVisible(); | ||
| await expect( | ||
| coverageCard.getByText('SonarQube coverage metrics'), | ||
| ).toBeVisible(); | ||
| await expect( | ||
| scorecardPage.getBucketTile(coverageCard, 'warning'), | ||
| ).toContainText('2'); | ||
| await expect( | ||
| scorecardPage.getBucketTile(coverageCard, 'success'), | ||
| ).toContainText('1'); | ||
|
|
||
| await runAccessibilityTests(page, testInfo); | ||
| }); | ||
|
|
||
| test('Verify data sources dialog opens from menu', async ({}, testInfo) => { | ||
| const securityCard = scorecardPage.getGroupCard('Security Vulnerabilities'); | ||
| const dialog = await scorecardPage.openDataSourcesDialog(securityCard); | ||
|
|
||
| await expect(dialog).toContainText( | ||
| scorecardPage.getDialogTitle('Security Vulnerabilities'), | ||
| ); | ||
|
|
||
| const expectedColumns = [ | ||
| translations.dataSourcesDialog.columns.plugin, | ||
| translations.dataSourcesDialog.columns.check, | ||
| translations.dataSourcesDialog.columns.value, | ||
| translations.dataSourcesDialog.columns.status, | ||
| translations.dataSourcesDialog.columns.lastSynced, | ||
| ]; | ||
| for (const column of expectedColumns) { | ||
| await expect(dialog.getByText(column, { exact: true })).toBeVisible(); | ||
| } | ||
|
|
||
| await runAccessibilityTests(page, testInfo, undefined, { | ||
| disableRules: ['color-contrast'], | ||
| }); | ||
|
|
||
| await scorecardPage.closeDialog(dialog); | ||
| await expect(dialog).not.toBeVisible(); | ||
| }); | ||
|
|
||
| test('Verify filter pills filter data sources by threshold in SonarQube Coverage', async () => { | ||
| const coverageCard = scorecardPage.getGroupCard('SonarQube Coverage'); | ||
| const dialog = await scorecardPage.openDataSourcesDialog(coverageCard); | ||
| const tableRows = scorecardPage.getTableRows(dialog); | ||
|
|
||
| await expect(tableRows).toHaveCount(3); | ||
|
|
||
| const warningPill = scorecardPage.getFilterPill(dialog, 'warning'); | ||
| await warningPill.click(); | ||
| await expect(warningPill).toHaveAttribute('aria-pressed', 'true'); | ||
| await expect(tableRows).toHaveCount(2); | ||
| for (const row of await tableRows.all()) { | ||
| await expect(row).toContainText(translations.thresholds.warning); | ||
| } | ||
|
|
||
| await warningPill.click(); | ||
| await expect(warningPill).toHaveAttribute('aria-pressed', 'false'); | ||
| await expect(tableRows).toHaveCount(3); | ||
|
|
||
| const successPill = scorecardPage.getFilterPill(dialog, 'success'); | ||
| await successPill.click(); | ||
| await expect(successPill).toHaveAttribute('aria-pressed', 'true'); | ||
| await expect(tableRows).toHaveCount(1); | ||
| for (const row of await tableRows.all()) { | ||
| await expect(row).toContainText(translations.thresholds.success); | ||
|
jrichter1 marked this conversation as resolved.
|
||
| } | ||
|
|
||
| await scorecardPage.closeDialog(dialog); | ||
| }); | ||
|
|
||
| test('Verify clicking bucket tile opens dialog with filter pre-applied', async () => { | ||
| const coverageCard = scorecardPage.getGroupCard('SonarQube Coverage'); | ||
|
|
||
| await scorecardPage.getBucketTile(coverageCard, 'warning').click(); | ||
|
|
||
| const dialog = page.locator('[role="dialog"]'); | ||
| await expect(dialog).toBeVisible({ timeout: 5000 }); | ||
|
|
||
| const activePill = dialog.locator('[role="button"][aria-pressed="true"]'); | ||
| await expect(activePill).toBeVisible(); | ||
| const tableRows = scorecardPage.getTableRows(dialog); | ||
| await expect(tableRows).toHaveCount(2); | ||
| for (const row of await tableRows.all()) { | ||
| await expect(row).toContainText(translations.thresholds.warning); | ||
| } | ||
|
|
||
| await scorecardPage.closeDialog(dialog); | ||
| }); | ||
|
|
||
| test('Verify ungrouped metrics render as individual cards alongside group cards', async ({}, testInfo) => { | ||
| await mockSonarqubeScorecardResponse(page, sonarqubeScorecardResponse); | ||
| await catalogPage.openCatalog(); | ||
| await catalogPage.openComponent('sonarqube-scorecard-only'); | ||
| await page.getByText('Scorecard', { exact: true }).click(); | ||
|
|
||
| const groupedMetricIds = new Set([ | ||
| 'sonarqube.qualityGate', | ||
| 'sonarqube.codeCoverage', | ||
| 'sonarqube.maintainabilityRating', | ||
| 'sonarqube.openIssues', | ||
| 'sonarqube.securityHotspots', | ||
| 'sonarqube.securityRating', | ||
| ]); | ||
|
|
||
| const ungroupedMetricTitles = Object.entries(translations.metric) | ||
| .filter( | ||
|
teknaS47 marked this conversation as resolved.
|
||
| ([key]) => key.startsWith('sonarqube.') && !groupedMetricIds.has(key), | ||
| ) | ||
| .map(([, value]) => (value as { title: string }).title); | ||
|
|
||
| for (const title of ungroupedMetricTitles) { | ||
| await expect(page.getByText(title, { exact: true }).first()).toBeVisible({ | ||
| timeout: 10000, | ||
| }); | ||
| } | ||
|
|
||
| await expect(page.getByLabel('Security Vulnerabilities')).toBeVisible(); | ||
| await expect(page.getByLabel('Code Quality')).toBeVisible(); | ||
| await expect(page.getByLabel('SonarQube Coverage')).toBeVisible(); | ||
|
|
||
| await runAccessibilityTests(page, testInfo); | ||
| }); | ||
|
|
||
| test('Verify grouped metric values are reflected in Code Quality group card', async () => { | ||
| await mockSonarqubeScorecardResponse(page, sonarqubeScorecardResponse); | ||
| await catalogPage.openCatalog(); | ||
|
teknaS47 marked this conversation as resolved.
|
||
| await catalogPage.openComponent('sonarqube-scorecard-only'); | ||
| await page.getByText('Scorecard', { exact: true }).click(); | ||
|
|
||
| await expect(page.getByLabel('Code Quality')).toBeVisible({ | ||
| timeout: 10000, | ||
| }); | ||
| }); | ||
|
|
||
| test('Verify quality gate failure shows error bucket on Code Quality group card', async () => { | ||
| await mockSonarqubeScorecardResponse( | ||
| page, | ||
| sonarqubeFailedQualityGateResponse, | ||
|
jrichter1 marked this conversation as resolved.
|
||
| ); | ||
| await catalogPage.openCatalog(); | ||
| await catalogPage.openComponent('sonarqube-scorecard-only'); | ||
| await page.getByText('Scorecard', { exact: true }).click(); | ||
|
|
||
| const codeQualityCard = page | ||
| .locator('[role="article"]') | ||
| .filter({ hasText: 'Code Quality' }) | ||
| .first(); | ||
| await expect(codeQualityCard).toBeVisible({ timeout: 10000 }); | ||
|
|
||
| const errorBucket = codeQualityCard | ||
| .locator('[role="button"]') | ||
| .filter({ hasText: translations.thresholds.error }); | ||
| await expect(errorBucket).toBeVisible(); | ||
| }); | ||
| }); | ||
Oops, something went wrong.
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.