Run automation tests on HyperExecute using Puppeteer-CodeceptJs
HyperExecute is a smart test orchestration platform to run end-to-end Puppeteer tests at the fastest speed possible. HyperExecute lets you achieve an accelerated time to market by providing a test infrastructure that offers optimal speed, test orchestration, and detailed execution logs.
The overall experience helps teams test code and fix issues at a much faster pace. HyperExecute is configured using a YAML file. Instead of moving the Hub close to you, HyperExecute brings the test scripts close to the Hub! This guide will cover the basics of getting started with Puppeteer-Codecept testing on the Lambdatest HyperExecute grid.
HyperExecute has several state of the art features to help you optmize your testing process. Go through the features page to take a look at all the tools that HyperExecute offers.
HyperExecute is compliant with leading security standards - SOC2, GDPR, and CCPA. Refer to HyperExecute Getting Started Guide for more information about features offered by HyperExecute.
All the code samples in this documentation can be found in the Puppeteer-Codecept HyperExecute GitHub repository. You can either download or clone the repository to run tests on the HyperExecute Grid.
Gitpod
Follow the below steps to run Gitpod button:
- Click the Open in Gitpod button (You will be redirected to Login/Signup page).
- Login with Lambdatest credentials. You will be redirected to HyperExecute dashboard with pop-up confirming to 'Proceed' to Gitpod editor in the new tab and current tab will show hyperexecute dashboard.
Prerequisites for running Puppeteer-CodeceptJs tests on HyperExecute Grid
- Download the HyperExecute CLI: The HyperExecute CLI is used for triggering tests on HyperExecute Grid. It provides a host of other useful features that accelerate test execution. You should download the HyperExecute CLI binary on the host system for running tests on HyperExecute. Shown below is the HyperExecute CLI download location for different platforms:
| Platform | HyperExecute CLI download location |
|---|---|
| Windows | https://downloads.lambdatest.com/hyperexecute/windows/hyperexecute.exe |
| macOS | https://downloads.lambdatest.com/hyperexecute/darwin/hyperexecute |
| Linux | https://downloads.lambdatest.com/hyperexecute/linux/hyperexecute |
For detailed information about HyperExecute CLI, please refer to HyperExecute CLI section in the HyperExecute getting started guide.
- Configure Environment Variables: Before the tests are run, please set the environment variables LT_USERNAME & LT_ACCESS_KEY from the terminal. The account details are available on your LambdaTest Profile page. For macOS:
export LT_USERNAME=LT_USERNAME
export LT_ACCESS_KEY=LT_ACCESS_KEY
For Linux:
export LT_USERNAME=LT_USERNAME
export LT_ACCESS_KEY=LT_ACCESS_KEY
For Windows:
set LT_USERNAME=LT_USERNAME
set LT_ACCESS_KEY=LT_ACCESS_KEY
Auto-Split Execution with Puppeteer-CodeceptJs
Auto-split execution mechanism lets you run tests at predefined concurrency and distribute the tests over the available infrastructure. Concurrency can be achieved at different levels - file, module, test suite, test, scenario, etc. For more information about auto-split execution, check out the Auto-Split Getting Started Guide
Core
Auto-split YAML file (yaml/win/.hyperexecute_autosplits.yaml) in the repo contains the following configuration:
globalTimeout: 90
testSuiteTimeout: 90
testSuiteStep: 90
Global timeout, testSuite timeout, and testSuite timeout are set to 90 minutes. The runson key determines the platform (or operating system) on which the tests are executed. Here we have set the target OS as Windows.
runson: win
Auto-split is set to true in the YAML file.
autosplit: true
retryOnFailure is set to true, instructing HyperExecute to retry failed command(s). The retry operation is carried out till the number of retries mentioned in maxRetries are exhausted or the command execution results in a Pass. In addition, the concurrency (i.e. number of parallel sessions) is set to 2.
retryOnFailure: true
runson: win
maxRetries: 2
Pre Steps and Dependency Caching
To leverage the advantage offered by Dependency Caching in HyperExecute, the integrity of package-lock.json is checked using the checksum functionality.
cacheKey: '{{ checksum "package-lock.json" }}'
The caching advantage offered by NPM can be leveraged in HyperExecute, whereby the downloaded packages can be stored (or cached) in a secure server for future executions. The packages available in the cache will only be used if the checksum stage results in a Pass.
cacheDirectories:
- node_modules
The testDiscovery directive contains the command that gives details of the mode of execution, along with detailing the command that is used for test execution. Here, we are fetching the list of Test file scenario that would be further executed using the value passed in the testRunnerCommand
testDiscovery:
type: raw
mode: dynamic
command: grep -lr 'Scenario' *test.js
testRunnerCommand: npx codeceptjs run $test --steps
Running the above command on the terminal will give a list of Test Scenario lines that are located in the Project folder: Test Discovery Output: react_app_test.js todo_test.js The testRunnerCommand contains the command that is used for triggering the test. The output fetched from the testDiscoverer command acts as an input to the testRunner command.
testRunnerCommand: npx codeceptjs run $test --steps
Artifacts Management
The mergeArtifacts directive (which is by default false) is set to true for merging the artifacts and combing artifacts generated under each task. The uploadArtefacts directive informs HyperExecute to upload artifacts [files, reports, etc.] generated after task completion. In the example, path consists of a regex for parsing the directory (i.e. reports that contains the test reports).
mergeArtifacts: true
uploadArtefacts:
[{
"name": "Reports",
"path": ["Reports\\"]
}]
HyperExecute also facilitates the provision to download the artifacts on your local machine. To download the artifacts, click on Artifacts button corresponding to the associated TestID.
Test Execution
The CLI option --config is used for providing the custom HyperExecute YAML file (i.e. yaml/.hyperexecute_autosplits.yaml). Run the following command on the terminal to trigger the tests in JS files on the HyperExecute grid. The --download-artifacts option is used to inform HyperExecute to download the artifacts for the job.
./hyperexecute --config --verbose yaml/win/.hyperexecute_autosplits.yaml
Visit HyperExecute Automation Dashboard to check the status of execution
Matrix Execution with Puppeteer-CodeceptJs
Matrix multiplexing execution strategy is used for running the same tests across different test (or input) combinations. The Matrix directive in HyperExecute YAML file is a key:value pair where value is an array of strings. Also, the key:value pairs are opaque strings for HyperExecute. For more information about matrix multiplexing, check out the Matrix Getting Started Guide
Core
In the current example, matrix YAML file (yaml/hyperexecute_matrix_sample.yaml) in the repo contains the following configuration:
globalTimeout: 90
testSuiteTimeout: 90
testSuiteStep: 90
Global timeout, testSuite timeout, and testSuite timeout are set to 90 minutes. The target platform is set to Windows. Please set the [runson] key to [mac] if the tests have to be executed on the macOS platform.
runson: win
Puppeteer-CodeceptJs js files in the 'Tests' folder contain the Test Scenario run on the HyperExecute grid. In the example, the Test file tests/test_4.spec.js run in parallel on the basis of scenario by using the specified input combinations.
matrix:
os: [linux]
files: ["react_app_test.js","todo_test.js"]
The testSuites object contains a list of commands (that can be presented in an array). In the current YAML file, commands for executing the tests are put in an array (with a '-' preceding each item). The npx command is used to run tests in .js files. The tags are mentioned as an array to the methods key that is a part of the matrix.
testSuites:
- npx codeceptjs run $files --steps
Pre Steps and Dependency Caching
Dependency caching is enabled in the YAML file to ensure that the package dependencies are not downloaded in subsequent runs. The first step is to set the Key used to cache directories.
cacheKey: '{{ checksum "package-lock.json" }}'
Set the array of files & directories to be cached. In the example, all the packages will be cached in the CacheDir directory.
cacheDirectories:
- node_modules
Steps (or commands) that must run before the test execution are listed in the pre run step. In the example, the packages listed in requirements.txt are installed using the npm install command.
pre:
- npm install
Artifacts Management
The mergeArtifacts directive (which is by default false) is set to true for merging the artifacts and combing artifacts generated under each task. The uploadArtefacts directive informs HyperExecute to upload artifacts [files, reports, etc.] generated after task completion. In the example, path consists of a regex for parsing the directory (i.e. reports that contains the test reports).
mergeArtifacts: true
uploadArtefacts:
[{
"name": "Reports",
"path": ["Reports\\"]
}]
HyperExecute also facilitates the provision to download the artifacts on your local machine. To download the artifacts, click on Artifacts button corresponding to the associated TestID.
Test Execution
The CLI option --config is used for providing the custom HyperExecute YAML file (i.e. yaml/win/.hyperexecute_matrix.yaml). Run the following command on the terminal to trigger the tests in Test file Scenario on the HyperExecute grid.
./hyperexecute --config --verbose yaml/win/.hyperexecute_matrix.yaml
Visit HyperExecute Automation Dashboard to check the status of execution:
Run Puppeteer-CodeceptJs tests on Windows, Linux and Mac platforms
The CLI option --config is used for providing the custom HyperExecute YAML file (i.e. yaml/win/.hyperexecute_autosplits.yaml for Windows and yaml/linux/.hyperexecute_autosplits.yaml for Linux). Run the following command on the terminal to trigger tests on Windows platform:
./hyperexecute --config --verbose yaml/win/.hyperexecute_autosplits.yaml
Run the following command on the terminal to trigger tests on Linux platform:
./hyperexecute --config --verbose yaml/linux/.hyperexecute_autosplits.yaml
Run the following command on the terminal to trigger tests on Mac platform:
./hyperexecute --config --verbose yaml/mac/.hyperexecute_autosplits.yaml
Secrets Management
In case you want to use any secret keys in the YAML file, the same can be set by clicking on the Secrets button on the dashboard.
All you need to do is create an environment variable that uses the secret key:
env:
AccessKey: ${{.secrets.AccessKey}}
Navigation in Automation Dashboard
HyperExecute lets you navigate from/to Test Logs in Automation Dashboard from/to HyperExecute Logs. You also get relevant get relevant Puppeteer test details like video, network log, commands, Exceptions & more in the Dashboard. Effortlessly navigate from the automation dashboard to HyperExecute logs (and vice-versa) to get more details of the test execution.
For any query or doubt, please feel free to contact us via 24×7 chat support or you can also drop a mail to support@lambdatest.com.
Happy testing!