-
Notifications
You must be signed in to change notification settings - Fork 1
Add BrowserStack SDK + Playwright xUnit/Reqnroll sample #1
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Open
Jimesh-browserstack
wants to merge
2
commits into
main
Choose a base branch
from
browserstack-sdk
base: main
Could not load branches
Branch not found: {{ refName }}
Loading
Could not load tags
Nothing to show
Loading
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline,
and old review comments may become outdated.
Open
Changes from all commits
Commits
Show all changes
2 commits
Select commit
Hold shift + click to select a range
File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,123 @@ | ||
| # Sanity workflow that verifies the xUnit + Reqnroll + Playwright BrowserStack | ||
| # SDK sample against a full commit id, mirroring browserstack/csharp-playwright-browserstack. | ||
| # Two test runs: | ||
| # 1. Public bstackdemo scenario (browserstackLocal: false in yml). | ||
| # 2. BrowserStack Local scenario (yml flipped to true; a python http.server | ||
| # hosts a tiny title-matching page on port 45454, the SDK starts the tunnel, | ||
| # and the test asserts that the cloud browser sees that page through bs-local.com). | ||
|
|
||
| name: xUnit Reqnroll Playwright SDK sanity workflow on workflow_dispatch | ||
|
|
||
| on: | ||
| workflow_dispatch: | ||
| inputs: | ||
| commit_sha: | ||
| description: 'The full commit id to build' | ||
| required: true | ||
|
|
||
| permissions: | ||
| contents: read | ||
| checks: write | ||
|
|
||
| jobs: | ||
| sanity: | ||
| runs-on: ${{ matrix.os }} | ||
| strategy: | ||
| fail-fast: false | ||
| max-parallel: 1 | ||
| matrix: | ||
| dotnet: ['8.0.x'] | ||
| os: [windows-latest] | ||
| name: xUnit Repo ${{ matrix.dotnet }} - ${{ matrix.os }} Sample | ||
| env: | ||
| BROWSERSTACK_USERNAME: ${{ secrets.BROWSERSTACK_USERNAME }} | ||
| BROWSERSTACK_ACCESS_KEY: ${{ secrets.BROWSERSTACK_ACCESS_KEY }} | ||
|
|
||
| steps: | ||
| - uses: actions/checkout@v3 | ||
| with: | ||
| ref: ${{ github.event.inputs.commit_sha }} | ||
|
|
||
| - uses: actions/github-script@98814c53be79b1d30f795b907e553d8679345975 | ||
| id: status-check-in-progress | ||
| env: | ||
| job_name: xUnit Repo ${{ matrix.dotnet }} - ${{ matrix.os }} Sample | ||
| commit_sha: ${{ github.event.inputs.commit_sha }} | ||
| with: | ||
| github-token: ${{ github.token }} | ||
| script: | | ||
| const result = await github.rest.checks.create({ | ||
| owner: context.repo.owner, | ||
| repo: context.repo.repo, | ||
| name: process.env.job_name, | ||
| head_sha: process.env.commit_sha, | ||
| status: 'in_progress' | ||
| }).catch((err) => ({status: err.status, response: err.response})); | ||
| console.log(`The status-check response : ${result.status} Response : ${JSON.stringify(result.response)}`) | ||
| if (result.status !== 201) { | ||
| console.log('Failed to create check run') | ||
| } | ||
|
|
||
| - name: Setup dotnet | ||
| uses: actions/setup-dotnet@v3 | ||
| with: | ||
| dotnet-version: ${{ matrix.dotnet }} | ||
|
|
||
| - name: Strip credential placeholders so env vars take effect | ||
| # The yml ships with literal YOUR_USERNAME / YOUR_ACCESS_KEY placeholders; | ||
| # the .NET SDK only falls back to env vars when those lines are absent. | ||
| shell: bash | ||
| working-directory: XunitReqnrollPlaywrightBrowserstack.Tests | ||
| run: | | ||
| sed -i '/^userName:/d; /^accessKey:/d' browserstack.yml | ||
|
|
||
| - name: Install dependencies | ||
| run: dotnet build | ||
|
|
||
| - name: Run sample tests (public bstackdemo) | ||
| working-directory: XunitReqnrollPlaywrightBrowserstack.Tests | ||
| run: dotnet test --filter "FullyQualifiedName~BStackDemoCart" | ||
|
|
||
| - name: Run local tests (BrowserStack Local + python http.server harness) | ||
| shell: bash | ||
| working-directory: XunitReqnrollPlaywrightBrowserstack.Tests | ||
| run: | | ||
| set -u | ||
| # 1. Stand up a tiny static page with a known <title>. | ||
| mkdir -p "$RUNNER_TEMP/bs-local-harness" | ||
| cat > "$RUNNER_TEMP/bs-local-harness/index.html" <<'HTML' | ||
| <!doctype html> | ||
| <html><head><title>BrowserStack Local Test</title></head> | ||
| <body>OK</body></html> | ||
| HTML | ||
| ( cd "$RUNNER_TEMP/bs-local-harness" && python -m http.server 45454 ) & | ||
| HTTP_PID=$! | ||
| trap 'kill "$HTTP_PID" 2>/dev/null || true' EXIT | ||
| sleep 2 | ||
| # 2. Flip the SDK Local toggle so the SDK starts/stops the tunnel. | ||
| sed -i 's/^browserstackLocal: false/browserstackLocal: true/' browserstack.yml | ||
| # 3. Run only the local scenario; cloud browser reaches the harness through bs-local.com. | ||
| dotnet test --filter "FullyQualifiedName~BStackLocalSample" | ||
|
|
||
| - if: always() | ||
| uses: actions/github-script@98814c53be79b1d30f795b907e553d8679345975 | ||
| id: status-check-completed | ||
| env: | ||
| conclusion: ${{ job.status }} | ||
| job_name: xUnit Repo ${{ matrix.dotnet }} - ${{ matrix.os }} Sample | ||
| commit_sha: ${{ github.event.inputs.commit_sha }} | ||
| with: | ||
| github-token: ${{ github.token }} | ||
| script: | | ||
| const result = await github.rest.checks.create({ | ||
| owner: context.repo.owner, | ||
| repo: context.repo.repo, | ||
| name: process.env.job_name, | ||
| head_sha: process.env.commit_sha, | ||
| status: 'completed', | ||
| conclusion: process.env.conclusion | ||
| }).catch((err) => ({status: err.status, response: err.response})); | ||
| console.log(`The status-check response : ${result.status} Response : ${JSON.stringify(result.response)}`) | ||
| if (result.status !== 201) { | ||
| console.log('Failed to create check run') | ||
| } |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,16 @@ | ||
| bin/ | ||
| obj/ | ||
| log/ | ||
| TestResults/ | ||
| .vs/ | ||
| .vscode/ | ||
| .idea/ | ||
| .config/ | ||
| .browserstack/ | ||
| *.user | ||
| *.suo | ||
| .DS_Store | ||
| browserstack.err | ||
|
|
||
| # Reqnroll-generated code-behind for .feature files | ||
| **/Features/*.feature.cs |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -1,2 +1,97 @@ | ||
| # xunit-reqnroll-playwright-browserstack | ||
| Sample repo for customers | ||
|
|
||
| This sample shows how to run [xUnit](https://xunit.net/) + [Reqnroll](https://reqnroll.net/) + [Playwright](https://playwright.dev/dotnet) tests on BrowserStack using the [BrowserStack .NET SDK](https://www.nuget.org/packages/BrowserStack.TestAdapter). The SDK reads `browserstack.yml`, fans your scenarios out across the platforms listed there, starts and stops BrowserStack Local automatically, and reports test status to the BrowserStack dashboard. Your test code stays pure `Microsoft.Playwright` + Reqnroll -- no manual `ConnectAsync`, no caps in code. | ||
|
|
||
|  | ||
|
|
||
| ## Run Sample Build | ||
|
|
||
| * Clone the repo | ||
| * Open the solution `XunitReqnrollPlaywrightBrowserstack.sln` in Visual Studio (or your IDE of choice) | ||
| * Build the solution (`dotnet build`) | ||
| * Replace the `userName` and `accessKey` placeholders in `browserstack.yml` with your [BrowserStack Username and Access Key](https://www.browserstack.com/accounts/settings). Alternatively, remove those two lines from the yml and set `BROWSERSTACK_USERNAME` and `BROWSERSTACK_ACCESS_KEY` as environment variables -- the SDK falls back to env vars only when the yml fields are absent | ||
|
|
||
| ### Running your tests from CLI | ||
|
|
||
| ```sh | ||
| cd XunitReqnrollPlaywrightBrowserstack.Tests | ||
| dotnet test | ||
| ``` | ||
|
|
||
| The sample runs across both platforms declared in `browserstack.yml` (Windows 11 / Chrome and macOS / WebKit) in parallel. | ||
|
|
||
| Understand how many parallel sessions you need by using our [Parallel Test Calculator](https://www.browserstack.com/automate/parallel-calculator?ref=github). | ||
|
|
||
| ### Testing a private host (BrowserStack Local) | ||
|
|
||
| If your app lives on `localhost`, a staging host, or behind a firewall, set `browserstackLocal: true` in `browserstack.yml` and rerun `dotnet test`. The SDK starts and stops the BrowserStack Local tunnel for you -- no manual binary download or lifecycle management. Then point your scenarios at `http://bs-local.com:<port>/` (a hostname BrowserStack Local resolves to your machine) instead of a public URL. | ||
|
|
||
| ## Integrate your test suite | ||
|
|
||
| This repository uses the BrowserStack SDK to run tests on BrowserStack. To wire the SDK into your own test suite: | ||
|
|
||
| * Create a `browserstack.yml` at the project root with your BrowserStack credentials and platform list (see this repo for a working template) | ||
| * Add the `BrowserStack.TestAdapter` NuGet package: | ||
|
|
||
| ```sh | ||
| dotnet add package BrowserStack.TestAdapter | ||
| ``` | ||
|
|
||
| * Build the project (`dotnet build`); the SDK installs the `browserstack-sdk` dotnet tool and patches the test assembly so Playwright launches are routed to BrowserStack at runtime | ||
|
|
||
| ## How the SDK changes things | ||
|
|
||
| - **One `browserstack.yml`** declares platforms, parallelism, the Local toggle, and reporting; the SDK picks them up automatically | ||
| - **The SDK runs platforms in parallel for you** -- one xUnit run per `(platform x parallelsPerPlatform)` cell, no per-platform branching needed | ||
| - **The SDK rewrites Playwright launches** -- `Hooks/PlaywrightHooks.cs` calls `pw.Chromium.LaunchAsync()` and the SDK transparently redirects to the per-platform browser configured in the yml (`chrome` / `playwright-webkit` / `playwright-firefox` / etc.). No `Chromium.ConnectAsync(wss_url)` plumbing | ||
| - **The SDK starts and stops BrowserStack Local** when `browserstackLocal: true` -- no manual tunnel lifecycle management | ||
| - **Reqnroll generates xUnit test classes** from `.feature` files at build time, so behaviour-driven scenarios run as standard xUnit tests under `dotnet test` | ||
|
|
||
| ## Repo layout | ||
|
|
||
| ``` | ||
| . | ||
| ├── XunitReqnrollPlaywrightBrowserstack.sln | ||
| └── XunitReqnrollPlaywrightBrowserstack.Tests/ | ||
| ├── XunitReqnrollPlaywrightBrowserstack.Tests.csproj | ||
| ├── browserstack.yml # SDK config: credentials, platforms, Local toggle, reporting | ||
| ├── Features/ | ||
| │ └── Sample.feature # bstackdemo add-to-cart scenario | ||
| ├── StepDefinitions/ | ||
| │ └── SampleSteps.cs | ||
| └── Hooks/ | ||
| └── PlaywrightHooks.cs # creates IPage per scenario; SDK routes the launch | ||
| ``` | ||
|
|
||
| ## Notes | ||
|
|
||
| * You can view your test results on the [BrowserStack Automate dashboard](https://www.browserstack.com/automate) | ||
| * To test on a different set of browsers, see our [list of supported browsers and platforms](https://www.browserstack.com/list-of-browsers-and-platforms?product=automate) | ||
| * You can export the environment variables for the Username and Access Key of your BrowserStack account: | ||
|
|
||
| * For Unix-like or Mac machines: | ||
| ```sh | ||
| export BROWSERSTACK_USERNAME=<browserstack-username> && | ||
| export BROWSERSTACK_ACCESS_KEY=<browserstack-access-key> | ||
| ``` | ||
| * For Windows Cmd: | ||
| ```cmd | ||
| set BROWSERSTACK_USERNAME=<browserstack-username> | ||
| set BROWSERSTACK_ACCESS_KEY=<browserstack-access-key> | ||
| ``` | ||
| * For Windows Powershell: | ||
| ```powershell | ||
| $env:BROWSERSTACK_USERNAME=<browserstack-username> | ||
| $env:BROWSERSTACK_ACCESS_KEY=<browserstack-access-key> | ||
| ``` | ||
|
|
||
| ## Further Reading | ||
|
|
||
| - [xUnit](https://xunit.net/) | ||
| - [Reqnroll](https://reqnroll.net/) | ||
| - [Playwright .NET](https://playwright.dev/dotnet/) | ||
| - [BrowserStack documentation for Playwright in C#](https://www.browserstack.com/docs/automate/playwright/getting-started/c-sharp) | ||
| - [BrowserStack.TestAdapter on NuGet](https://www.nuget.org/packages/BrowserStack.TestAdapter) | ||
| - [NUnit reference sample](https://github.com/browserstack/csharp-playwright-browserstack) | ||
|
|
||
| Happy Testing! |
9 changes: 9 additions & 0 deletions
9
XunitReqnrollPlaywrightBrowserstack.Tests/Features/LocalSample.feature
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,9 @@ | ||
| Feature: BStackLocalSample | ||
|
|
||
| As a developer testing a private host | ||
| I want BrowserStack Local to tunnel my localhost to BrowserStack | ||
| So that the cloud browser can reach a page only my machine can serve | ||
|
|
||
| Scenario: Reach a private host via BrowserStack Local | ||
| Given I open the local sample page on bs-local | ||
| Then the local sample page title contains "BrowserStack Local" |
10 changes: 10 additions & 0 deletions
10
XunitReqnrollPlaywrightBrowserstack.Tests/Features/Sample.feature
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,10 @@ | ||
| Feature: BStackDemo cart | ||
|
|
||
| As a shopper on bstackdemo.com | ||
| I want to add an item to my cart | ||
| So that I can verify the cart shows what I picked | ||
|
|
||
| Scenario: Add the first item to cart | ||
| Given I open the bstackdemo home page | ||
| When I add the first product to the cart | ||
| Then the cart shows 1 item that matches the product I added |
31 changes: 31 additions & 0 deletions
31
XunitReqnrollPlaywrightBrowserstack.Tests/Hooks/PlaywrightHooks.cs
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,31 @@ | ||
| using Microsoft.Playwright; | ||
| using Reqnroll; | ||
|
|
||
| namespace XunitReqnrollPlaywrightBrowserstack.Tests.Hooks; | ||
|
|
||
| [Binding] | ||
| public class PlaywrightHooks | ||
| { | ||
| private readonly ScenarioContext _scenario; | ||
| private IPlaywright? _pw; | ||
| private IBrowser? _browser; | ||
|
|
||
| public PlaywrightHooks(ScenarioContext scenario) => _scenario = scenario; | ||
|
|
||
| [BeforeScenario] | ||
| public async Task SetUp() | ||
| { | ||
| _pw = await Playwright.CreateAsync(); | ||
| _browser = await _pw.Chromium.LaunchAsync(); | ||
| var context = await _browser.NewContextAsync(); | ||
| var page = await context.NewPageAsync(); | ||
| _scenario.Set(page, "page"); | ||
| } | ||
|
|
||
| [AfterScenario] | ||
| public async Task TearDown() | ||
| { | ||
| if (_browser is not null) await _browser.CloseAsync(); | ||
| _pw?.Dispose(); | ||
| } | ||
| } | ||
28 changes: 28 additions & 0 deletions
28
XunitReqnrollPlaywrightBrowserstack.Tests/StepDefinitions/LocalSampleSteps.cs
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,28 @@ | ||
| using Microsoft.Playwright; | ||
| using Reqnroll; | ||
|
|
||
| namespace XunitReqnrollPlaywrightBrowserstack.Tests.StepDefinitions; | ||
|
|
||
| // Mirrors browserstack/csharp-playwright-browserstack -> SampleLocalTest.cs: | ||
| // page.GotoAsync("http://bs-local.com:45454/") + title.Contains("BrowserStack Local") | ||
| [Binding] | ||
| public class LocalSampleSteps | ||
| { | ||
| private readonly ScenarioContext _scenario; | ||
| private IPage Page => _scenario.Get<IPage>("page"); | ||
|
|
||
| public LocalSampleSteps(ScenarioContext scenario) => _scenario = scenario; | ||
|
|
||
| [Given(@"I open the local sample page on bs-local")] | ||
| public async Task OpenLocalSamplePage() | ||
| { | ||
| await Page.GotoAsync("http://bs-local.com:45454/"); | ||
| } | ||
|
|
||
| [Then(@"the local sample page title contains ""(.*)""")] | ||
| public async Task LocalSamplePageTitleContains(string expected) | ||
| { | ||
| var actual = await Page.TitleAsync(); | ||
| Assert.Contains(expected, actual); | ||
| } | ||
| } |
39 changes: 39 additions & 0 deletions
39
XunitReqnrollPlaywrightBrowserstack.Tests/StepDefinitions/SampleSteps.cs
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,39 @@ | ||
| using Microsoft.Playwright; | ||
| using Reqnroll; | ||
|
|
||
| namespace XunitReqnrollPlaywrightBrowserstack.Tests.StepDefinitions; | ||
|
|
||
| [Binding] | ||
| public class SampleSteps | ||
| { | ||
| private readonly ScenarioContext _scenario; | ||
| private IPage Page => _scenario.Get<IPage>("page"); | ||
| private string _productTitle = string.Empty; | ||
|
|
||
| public SampleSteps(ScenarioContext scenario) => _scenario = scenario; | ||
|
|
||
| [Given(@"I open the bstackdemo home page")] | ||
| public async Task OpenHome() | ||
| { | ||
| await Page.GotoAsync("https://bstackdemo.com/"); | ||
| } | ||
|
|
||
| [When(@"I add the first product to the cart")] | ||
| public async Task AddFirstProduct() | ||
| { | ||
| var firstProduct = Page.Locator("[id=\"\\31 \"]"); | ||
| var titles = await firstProduct.Locator(".shelf-item__title").AllInnerTextsAsync(); | ||
| _productTitle = titles[0]; | ||
| await firstProduct.GetByText("Add to Cart").ClickAsync(); | ||
| } | ||
|
|
||
| [Then(@"the cart shows 1 item that matches the product I added")] | ||
| public async Task CartHasOneMatchingItem() | ||
| { | ||
| var quantity = await Page.Locator(".bag__quantity").InnerTextAsync(); | ||
| Assert.Equal("1", quantity); | ||
|
|
||
| var cartTitle = await Page.Locator(".shelf-item__details").Locator(".title").InnerTextAsync(); | ||
| Assert.Equal(_productTitle, cartTitle); | ||
| } | ||
| } |
29 changes: 29 additions & 0 deletions
29
XunitReqnrollPlaywrightBrowserstack.Tests/XunitReqnrollPlaywrightBrowserstack.Tests.csproj
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,29 @@ | ||
| <Project Sdk="Microsoft.NET.Sdk"> | ||
|
|
||
| <PropertyGroup> | ||
| <TargetFramework>net8.0</TargetFramework> | ||
| <ImplicitUsings>enable</ImplicitUsings> | ||
| <Nullable>enable</Nullable> | ||
|
|
||
| <IsPackable>false</IsPackable> | ||
| <IsTestProject>true</IsTestProject> | ||
| </PropertyGroup> | ||
|
|
||
| <ItemGroup> | ||
| <PackageReference Include="BrowserStack.TestAdapter" Version="0.*" /> | ||
| <PackageReference Include="coverlet.collector" Version="6.0.0" /> | ||
| <PackageReference Include="Microsoft.NET.Test.Sdk" Version="17.8.0" /> | ||
| <PackageReference Include="Microsoft.Playwright" Version="*" /> | ||
| <PackageReference Include="Reqnroll.xUnit" Version="3.3.4" /> | ||
| <PackageReference Include="xunit" Version="2.9.2" /> | ||
| <PackageReference Include="xunit.runner.visualstudio" Version="2.8.2"> | ||
| <IncludeAssets>runtime; build; native; contentfiles; analyzers; buildtransitive</IncludeAssets> | ||
| <PrivateAssets>all</PrivateAssets> | ||
| </PackageReference> | ||
| </ItemGroup> | ||
|
|
||
| <ItemGroup> | ||
| <Using Include="Xunit" /> | ||
| </ItemGroup> | ||
|
|
||
| </Project> |
Oops, something went wrong.
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Is this SDK approach?