Run Playwright Automation Tests on HyperExecute Using .Net

Playwright is a Node.js library that uses a single API to automate Chromium, Firefox, and WebKit. It is designed to enable powerful, reliable, and efficient automated browser testing. Playwright can also automate Microsoft Edge since it is built on the open-source Chromium web framework.

LambdaTest allows you to run Playwright tests across 40+ real browsers and operating system combinations. This guide will cover the basics of getting started with Playwright testing using Vanilla Javascript on the Lambdatest HyperExecute grid.

HyperExecute has several state of the art features to help you optmize your testing process. Go through the features page to take a look at all the tools that HyperExecute offers.

HyperExecute is compliant with leading security standards - SOC2, GDPR, and CCPA. Refer to HyperExecute Getting Started Guide for more information about features offered by HyperExecute.

All the code samples in this documentation can be found in the Playwright .Net HyperExecute GitHub repository. You can either download or clone the repository to run tests on the HyperExecute Grid.

Gitpod

Follow the below steps to run Gitpod button:

Click the Open in Gitpod button (You will be redirected to Login/Signup page).
Login with Lambdatest credentials. You will be redirected to HyperExecute dashboard with pop-up confirming to 'Proceed' to Gitpod editor in the new tab and current tab will show hyperexecute dashboard.

Prerequisites for Running Playwright .Net Tests On The HyperExecute Grid

Download the HyperExecute CLI:
The HyperExecute CLI is used for interacting with, and running the tests on the HyperExecute Grid. It provides a host of other useful features that accelerate test execution. You need to download the HyperExecute CLI binary corresponding to the platform (or OS) from where the tests are triggered. Shown below is the HyperExecute CLI download location for different platforms:

Platform	HyperExecute CLI download location
Windows	https://downloads.lambdatest.com/hyperexecute/windows/hyperexecute.exe
macOS	https://downloads.lambdatest.com/hyperexecute/darwin/hyperexecute
Linux	https://downloads.lambdatest.com/hyperexecute/linux/hyperexecute

For detailed information about HyperExecute CLI, please refer to HyperExecute CLI section in the HyperExecute getting started guide.

Configure Environment Variables:
Before the tests are run, please set the environment variables LT_USERNAME & LT_ACCESS_KEY from the terminal. The account details are available on your LambdaTest Profile page. For macOS:

export LT_USERNAME=LT_USERNAME
export LT_ACCESS_KEY=LT_ACCESS_KEY

For Linux:

export LT_USERNAME=LT_USERNAME
export LT_ACCESS_KEY=LT_ACCESS_KEY

For Windows:

set LT_USERNAME=LT_USERNAME
set LT_ACCESS_KEY=LT_ACCESS_KEY

Auto-Split Execution with Playwright .Net

Auto-split execution mechanism lets you run tests at predefined concurrency and distribute the tests over the available infrastructure. Concurrency can be achieved at different levels - file, module, test suite, test, scenario, etc.

For more information about auto-split execution, check out the Auto-Split Getting Started Guide

Core

Auto-split YAML file (yaml/win/dotnet_playwright_hyperexecute_autosplit_sample.yaml) in the repo contains the following configuration:

globalTimeout: 150
testSuiteTimeout: 150
testSuiteStep: 150

Global timeout, testSuite timeout, and testSuite timeout are set to 150 minutes.

The runson key determines the platform (or operating system) on which the tests are executed. Here we have set the target OS as Windows.

runson: win

Auto-split is set to true in the YAML file.

 autosplit: true

retryOnFailure is set to true, instructing HyperExecute to retry failed command(s). The retry operation is carried out till the number of retries mentioned in maxRetries are exhausted or the command execution results in a Pass. In addition, the concurrency (i.e. number of parallel sessions) is set to 4.

retryOnFailure: true
maxRetries: 5
concurrency: 4

Pre Steps and Dependency Caching

Dependency caching is enabled in the YAML file to ensure that the package dependencies are not downloaded in subsequent runs. The first step is to set the Key used to cache directories. The directory m2_cache_dir is created in the project's root directory.

env:
  NUGET_PACKAGES: 'C:\nuget_global_cache'
  NUGET_HTTP_CACHE_PATH: 'C:\nuget_http_cache'
  NUGET_PLUGINS_CACHE_PATH: 'C:\nuget_plugins_cache'
  # Secrets in Vault
  # PAT: ${{ .secrets.testKey }}

cacheKey: '{{ checksum "packages.txt" }}'

Steps (or commands) that must run before the test execution are listed in the pre run step. In the example, the Maven packages are downloaded in the m2_cache_dir. To prevent test execution at the pre stage, the maven.test.skip parameter is set to true so that only packages are downloaded and no test execution is performed.

pre:
  - dotnet list PlaywrightDotnetTests.csproj package > packages.txt
  - nuget locals all -clear
  - dotnet build -c Release

Post Steps

Steps (or commands) that need to run after the test execution are listed in the post step. In the example, we cat the contents of yaml/win/dotnet_playwright_hyperexecute_autosplit_sample.yaml

post:
  - cat yaml/win/dotnet_playwright_hyperexecute_autosplit_sample.yaml

The testDiscovery directive contains the command that gives details of the mode of execution, along with detailing the command that is used for test execution. Here, we are fetching the list of test names that would be further passed in the testRunnerCommand

testDiscovery:
  type: raw
  mode: static
  command: grep "case" Program.cs | awk '{print$2}' | sed 's/://g'

Running the above command on the terminal will give a list of scenarios present in the Program file:

single
iphonetest
ipadtest

The testRunnerCommand contains the command that is used for triggering the test. The output fetched from the testDiscoverer command acts as an input to the testRunner command.

testRunnerCommand: dotnet run $test

Artifacts Management

The mergeArtifacts directive (which is by default false) is set to true for merging the artifacts and combing artifacts generated under each task.

The uploadArtefacts directive informs HyperExecute to upload artifacts [files, reports, etc.] generated after task completion. In the example, path consists of a regex for parsing the directories that contain the XML and JSON reports (i.e. target/surefire-reports/testng-results.xml and target/surefire-reports/testng-results.xml) directory.

mergeArtifacts: true

uploadArtefacts:
  - name: XmlReports
    path:
      - PlaywrightTestReport.xml
  - name: JsonReports
    path:
      - PlaywrightTestReport.json

Test Execution

The CLI option --config is used for providing the custom HyperExecute YAML file (i.e. yaml/win/dotnet_playwright_hyperexecute_autosplit_sample.yaml for Windows and yaml/linux/dotnet_playwright_hyperexecute_autosplit_sample.yaml for Linux).

Execute Playwright .Net tests using Autosplit mechanism on Windows platform

Run the following command on the terminal to trigger Playwright tests on the Windows platform. The --download-artifacts option is used to inform HyperExecute to download the artifacts for the job. The --force-clean-artifacts option force cleans any existing artifacts for the project.

./hyperexecute --config yaml/win/dotnet_playwright_hyperexecute_autosplit_sample.yaml --force-clean-artifacts --download-artifacts

Execute Playwright .Net tests using Autosplit mechanism on Linux platform

Run the following command on the terminal to trigger Playwright tests on the Linux platform. The --download-artifacts option is used to inform HyperExecute to download the artifacts for the job. The --force-clean-artifacts option force cleans any existing artifacts for the project.

./hyperexecute --config yaml/linux/dotnet_playwright_hyperexecute_autosplit_sample.yaml --force-clean-artifacts --download-artifacts

Visit HyperExecute Automation Dashboard to check the status of execution

Matrix Execution with Playwright .Net

Matrix-based test execution is used for running the same tests across different test (or input) combinations. The Matrix directive in HyperExecute YAML file is a key:value pair where value is an array of strings.

Also, the key:value pairs are opaque strings for HyperExecute. For more information about matrix multiplexing, check out the Matrix Getting Started Guide

Core

In the current example, matrix YAML file (yaml/win/dotnet_playwright_hyperexecute_matrix_sample.yaml) in the repo contains the following configuration:

globalTimeout: 150
testSuiteTimeout: 150
testSuiteStep: 150

Global timeout, testSuite timeout, and testSuite timeout are set to 150 minutes.

The target platform is set to Win. Please set the [runson] key to [mac] if the tests have to be executed on the macOS platform.

runson: win

The matrix constitutes of the following entries - Playwrighttag. The entries represent the scenario names in the feature files located in src/main/java/Features.

matrix:
  tests: ["single", "iphonetest", "ipadtest"]

The testSuites object contains a list of commands (that can be presented in an array). In the current YAML file, commands for executing the tests are put in an array (with a '-' preceding each item). The Maven command mvn test is used to run tests located in the current project. In the current project, parallel execution is achieved at the class level. The maven.repo.local parameter in Maven is used for overriding the location where the dependent Maven packages are downloaded. The Playwright.options parameter is used for filtering tests at the scenario level.

testSuites:
  - dotnet run $tests

Pre Steps and Dependency Caching

env:
  NUGET_PACKAGES: 'C:\nuget_global_cache'
  NUGET_HTTP_CACHE_PATH: 'C:\nuget_http_cache'
  NUGET_PLUGINS_CACHE_PATH: 'C:\nuget_plugins_cache'

cacheKey: '{{ checksum "packages.txt" }}'

pre:
  - dotnet list PlaywrightDotnetTests.csproj package > packages.txt
  - nuget locals all -clear
  - dotnet build -c Release

Post Steps

Steps (or commands) that need to run after the test execution are listed in the post step. In the example, we cat the contents of yaml/win/dotnet_playwright_hyperexecute_matrix_sample.yaml

post:
  - cat yaml/win/dotnet_playwright_hyperexecute_matrix_sample.yaml

Artifacts Management

The mergeArtifacts directive (which is by default false) is set to true for merging the artifacts and combing artifacts generated under each task.

mergeArtifacts: true

uploadArtefacts:
  - name: XmlReports
    path:
      - PlaywrightTestReport.xml
  - name: JsonReports
    path:
      - PlaywrightTestReport.json

Test Execution

The CLI option --config is used for providing the custom HyperExecute YAML file (i.e. yaml/win/dotnet_playwright_hyperexecute_matrix_sample.yaml for Windows and yaml/linux/dotnet_playwright_hyperexecute_matrix_sample.yaml for Linux).

Execute Playwright .Net tests using Matrix mechanism on Windows platform

./hyperexecute --config yaml/win/dotnet_playwright_hyperexecute_matrix_sample.yaml --force-clean-artifacts --download-artifacts

Execute Playwright .Net tests using Matrix mechanism on Linux platform

./hyperexecute --config yaml/linux/dotnet_playwright_hyperexecute_matrix_sample.yaml --force-clean-artifacts --download-artifacts

Visit HyperExecute Automation Dashboard to check the status of execution:

Secrets Management

If you want to use any secret keys in the YAML file, this can be set by clicking on the Secrets button on the dashboard.

All you need to do is create an environment variable that uses the secret key:

env:
  PAT: ${{ .secrets.testKey }}

For any query or doubt, please feel free to contact us via 24×7 chat support or you can also drop a mail to support@lambdatest.com.
Happy testing!

Run Playwright Automation Tests on HyperExecute Using .Net

Gitpod​

Prerequisites for Running Playwright .Net Tests On The HyperExecute Grid​

Download the HyperExecute CLI:​

Configure Environment Variables:​

Auto-Split Execution with Playwright .Net​

Core​

Pre Steps and Dependency Caching​

Post Steps​

Artifacts Management​

Test Execution​

Execute Playwright .Net tests using Autosplit mechanism on Windows platform​

Execute Playwright .Net tests using Autosplit mechanism on Linux platform​

Matrix Execution with Playwright .Net​

Core​

Pre Steps and Dependency Caching​

Post Steps​

Artifacts Management​

Test Execution​

Execute Playwright .Net tests using Matrix mechanism on Windows platform​

Execute Playwright .Net tests using Matrix mechanism on Linux platform​

Secrets Management​

Was this page helpful?

Gitpod

Prerequisites for Running Playwright .Net Tests On The HyperExecute Grid

Download the HyperExecute CLI:

Configure Environment Variables:

Auto-Split Execution with Playwright .Net

Core

Pre Steps and Dependency Caching

Post Steps

Artifacts Management

Test Execution

Execute Playwright .Net tests using Autosplit mechanism on Windows platform

Execute Playwright .Net tests using Autosplit mechanism on Linux platform

Matrix Execution with Playwright .Net

Core

Pre Steps and Dependency Caching

Post Steps

Artifacts Management

Test Execution

Execute Playwright .Net tests using Matrix mechanism on Windows platform

Execute Playwright .Net tests using Matrix mechanism on Linux platform

Secrets Management