How To Set Test Case Priority In TestNG With Selenium

TestNG is an open-source and powerful test automation framework with a wide range of features, including annotations, data providers, group testing, prioritization, and much more.

Prioritization in TestNG is an easy way to set test case priority and run your test case in the exact order you want. This feature is particularly useful when defining a sequence for the test case execution when you have multiple test cases or assign precedence to a few methods over others. For example, if you want your login scenario to run first, followed by adding an item to your cart, you don’t need to worry as TestNG has got you all covered.

In our previous blogs of this Selenium TestNG tutorial series, we have learned how to install TestNG, how to use TestNG Annotations, and much more. In this blog, we will cover how to set test case priority in TestNG while performing Selenium automation testing. If you are preparing for an interview you can learn more through TestNG interview questions.

Let’s get started!

What is test Priority in TestNG?

In TestNG, Priority is an attribute that helps the users define the order in which they want the test cases to be executed.

When you have multiple test cases and want them to run in a particular order, you can use the Priority attribute to set test priority in TestNG. The test cases get executed following an ascending order in the priority list, and hence, test cases with lower priority will always get executed first.

Syntax for using test Priority in TestNG

To set test case priority in TestNG, we need to add annotation as @Test (priority=X). In the below-shown example, we have given a priority of 1 to the test case.

What is default Priority in TestNG?

The default test priority in TestNG is 0. So, when you don’t set test priority in TestNG explicitly to the test cases, TestNG assigns them a priority of 0.

For example, the below-shown test case doesn’t have any priority set, and hence, it assumes a priority of 0.

With TestNG certification, you can challenge your skills in performing automated testing with TestNG and take your career to the next level.

Here’s a short glimpse of the TestNG certification from LambdaTest:

How to set negative Priority in TestNG?

If you wonder whether we can assign a negative priority value to any test case, the answer is Yes!! We can assign a negative priority to the test methods when we want them to take precedence over the default ones.

The below-shown example has two test methods, one with a default priority 0 and the other with negative priority, i.e., -1. Hence, the test method with negative priority will always be executed first, followed by the test method with default priority.

Console Output:

Also Read – How To Use DataProviders In TestNG [With Examples]

Watch this video to learn how to set up and use TestNG with Selenium to automate your testing process. We will also introduce the TestNG Priority method, which allows you to write easy-to-read and maintainable tests.

How to test with the same Priority in TestNG?

You must wonder what happens if you have a test case with all or some of the methods having the same priority. In such cases, TestNG follows an alphabetical order while executing them.

The below example has two test methods (a and b), and both have the same default priority that is 0. Therefore, the order of execution will be method a followed by b.

Console Output:

Also Read – Parameterization In TestNG – DataProvider and TestNG XML (With Examples)

Watch this video to learn how TestNG has become one of the most robust test automation frameworks and all you need to know to get started with TestNG in Selenium.

How to set test case Priority in TestNG using Selenium?

In this blog on how to set test case priority in TestNG, we have learned the fundamentals of priority in TestNG. In this section of the Selenium TestNG tutorial, we will see how test priority in TestNG works or how to set test case priority in TestNG. LambdaTest is an automated cross browser testing platform. With over 3000 online browser and OS combos, it lets you manage your manual and automated tests in one place with ease. Here’s how you can perform cross browser testing using TestNG.

Problem Scenario:

We will automate the main page of Selenium Playground offered by LamdaTest and print the headers of all the sections sequentially.

Implementation:

We will implement the problem scenario in four different ways, as listed below:

• When all the methods are without priority
• When all the methods are with priority
• When all the methods are with the same priority
• When there are both methods with and without priority

If you are getting started with running TestNG scripts using Selenium WebDriver, you can refer to our blog on Create TestNG Project In Eclipse & Run Selenium Test Script.

Implementing test case Priority when methods are without Priority in TestNG

In this section of the Selenium TestNG tutorial, we will see how to set test case priority in TestNG when all the methods are without any priority using a sample Selenium automation code.

You can use the below testng.xml file for running the above test class.

The below pom.xml can be used for installing all the necessary dependencies.

Also Read – How To Group Test Cases In TestNG [with Examples]

Code Walkthrough:

1. Imported Dependencies: Here, we have imported all the necessary classes, including the Desired Capabilities, Reporter Class, etc., for using the corresponding methods.

   Reporter Class is an inbuilt class in TestNG and is used for logging messages in the HTML Reports and Standard Output.

   You can refer to How to Use TestNG Reporter Class In Selenium that deep dives into the details of Reporter Class.

   

   Also Read: How To Generate TestNG Reports In Jenkins?

2. Global Variables: Here, we have used an online Selenium Grid like LamdaTest to perform parallel testing. Here is a brief video to help you with Parallel Testing in TestNG.

   Here, you can populate the values for your corresponding username and access key, which can be collected by logging into your LamdaTest Profile Section. You can copy the Username and the Access Token to be used in the code. However, the grid URL will remain the same, as shown below.

   

3. @BeforeClass(Setup Method): Here, we have used the LamdaTest Desired Capabilities Generator and have set the necessary capabilities of browser name, version, platform, etc., for our Selenium Remote Web Driver.

   Do refer to our detailed blog that deep dives into the important TestNG Annotations that are used for web automation testing.

   Along with setting the desired capabilities, we are logging into the Selenium Playground in the Before Class Method to use the platform for all the following test methods by logging in once.

   We are also logging the text message “Logging into Selenium Playground” in our HTML Report as well as standard output by making use of the Reporter Class Method.

   

4. @Test(getFirstOptionName): In this method, we are getting the Web Element of the header of the first section of the Selenium Playground highlighted below. Then, we are printing the text of the same Web Element, using Reporter Class on the console and HTML Report. Here, the text of the web element printed on the console will be Input Forms.

   
   Since there is no priority explicitly set, the test method will assume the default priority of 0.

   

   Here, we have made use of XPath in our test case for all the web elements. You can get the XPath of any element by simply doing a right-click on the Element→ Inspect Option. You can also refer to the complete guide on XPath in Selenium for understanding how you can make the most out of XPath for accessing WebElements in the DOM.

   

5. @Test(getSecondOptionName): In this method, we are getting the Web Element of the header of the second section of the Selenium Playground. Then, we are printing the text of the same Web Element, using Reporter Class on the console and HTML Report. Here, the text of the web element printed on the console will be Progress Bars & Sliders. The test method here again assumes the default priority of 0.
6. @Test(getThirdOptionName): In this method, we are getting the Web Element of the header of the third section of the Selenium Playground. Then, we are printing the text of the same Web Element, using Reporter Class on the console and HTML Report. Here, the text of the web element printed on the console will be Alerts & Modals. The test method assumes the default priority of 0.
7. @Test(getFourthOptionName): In this method, we are getting the Web Element of the header of the fourth section of the Selenium Playground. Then, we are printing the text of the same Web Element, using Reporter Class on the console and HTML Report. Here, the text of the web element printed on the console will be Date Pickers.
8. @Test(getFifthOptionName): In this method, we are getting the Web Element of the header of the fifth section of the Selenium Playground. Then, we are printing the text of the same Web Element, using Reporter Class on the console and HTML Report. Here, the text of the web element printed on the console will be Table.
9. @Test(getSixthOptionName): In this method, we are getting the Web Element of the header of the sixth section of the Selenium Playground. Then, we are printing the text of the same Web Element, using Reporter Class on the console and HTML Report. Here, the text of the web element printed on the console will be List Box.
10. @AfterClass(closeBrowser): In this method, we are simply closing the browser launched before class once all the test methods are executed.

Execution Results:

Since no priority was explicitly mentioned for all the above-shown test methods, TestNG will assign them a default priority of 0. Now, as explained in Tests With Same Priority In TestNG, to tie break between all these methods having the same priority, TestNG will follow alphabetical order while executing them. As a result, console output will look something like this:

You can further check the execution results of your test published at LambdaTest Automation Dashboard.

While your test is running, you can also see the live video streaming of your tests and various other details such as logs or exceptions raised.

Watch this video to learn about the TestNG Annotations and how they help provide better structure and readability to the code.

Implementing test case Priority when methods are with Priority in TestNG

In the previous section of this blog on how to set test case priority in TestNG, you saw the test methods getting executed alphabetically since they had a default priority assigned. Now, let’s see how the same test case would behave when we explicitly assign priority to the test methods while performing Selenium automation testing.

The below testng.xml file can be used for running the above Java class.

Code Walkthrough:

In this test case, the scenario remains the same as what we saw in our first example, except that all the methods have been assigned a priority here.

Execution Results:

Since all the test methods had their respective priorities assigned, TestNG will execute them in ascending order of their priorities. Hence, the method getFirstOptionName with priority=1 will be executed first, followed by getSecondOptionName with priority=2, and so on. As a result, console output looks like this:

You can further check the LamdaTest Automation Dashboard to check your test results and logs.

Implementing test case Priority when methods are with the same Priority in TestNG

In the previous section of this blog on how to set test case priority in TestNG, we saw how easily TestNG picked up our test methods based on their priority in ascending order, but what if the methods had the same priority assigned. So let’s see how our test result would look if the same test methods run with an equal priority assigned.

In this method, we will also assign a negative priority to a few of our test methods.

You can use the below testng.xml file for running the above Java class.

Code Walkthrough:

In this test case, the scenario remains the same as what we saw in our first and second examples, except that all the methods have been assigned a priority here, two of them having the same priority and few being assigned a negative priority.

Let’s decode how our results would look like. Here, the test method getSixthOptionName is assigned the lowest priority of -3, and hence, it will be picked up first, followed by getFifthOptionName with priority = -2, and then getFourthOptionName with priority = -1.

After these, there are two test methods getFirstOptionName and getSecondOptionName running with the same priority = 1. Hence, out of these two, getFirstOptionName will be executed first, since it comes first alphabetically followed by getSecondOptionName. The last method to be executed will be getThirdOptionName running with priority = 2.

Execution Results:

As expected, the console output will look like:

You can further check the LamdaTest Automation Dashboard to check your test results and logs.

Implementing test case Priority with both Prioritized and Non-Prioritized methods in TestNG

In the previous section of this blog on how to set test case priority in TestNG, we had either all the test methods without any priority set or all running with priority. But can we have a combination of both prioritized and non-prioritized methods in TestNG? The answer is YES.

Let’s explore a sample test case and understand how TestNG would execute our test methods in such scenarios.

You can use the below testng.xml for running the Java class.

Code Walkthrough:

In this test case, the scenario remains the same as what we saw until now, except that some methods have been assigned a priority here while some are running with default priority.

Let’s see how our results would look like, Here, the test methods getFifthOptionName has been assigned a priority =0, but it still falls at the same level as methods getFirstOptionName and getSecondOptionName since even though they are not assigned any priority explicitly, they will run with the default priority which is 0. Hence, out of all these three methods, getFifthOptionName will run first, followed by getFirstOptionName and getSecondOptionName, considering their alphabetical order. After these, the test method getFourthOptionName will be executed, followed by getThirdOptionName as both of them have the same priority set, which is 1. In the end, getSixthOptionName will be executed since it is running with a priority of 2.

Execution Results:

As expected, the console output will look like:

You can further check the LamdaTest Automation Dashboard to check your test results and logs.

TestNG issue when running tests with same Priority and in Parallel

We saw how to set case priority in TestNG and how the TestNG priority worked in different scenarios when they had the same or different priority set or even when they had no priority set. However, the scenarios where the same priority assigned methods could behave a little differently if you would run them in parallel.

You can refer to Create TestNG XML File & Execute Parallel Testing that deep dives into the details of parallel testing in TestNG using TestNG XML file.

For example, If we use our same test case where all the methods had no priority explicitly defined, which means they run with a default priority, i.e., 0 but, instead of running the test methods sequentially, let’s run them in parallel and see how the result would look like.

Let’s use the below testng.xml and run the same class in parallel with a thread count of 6.

Execution Results:

Now ideally, the test methods should have been executed in alphabetical order since they all had a default priority of 0 assigned. So, getFifthOptionName should run first, followed by getFirstOptionName, and so on.

Also Read – Speed Up Automated Parallel Testing In Selenium With TestNG

Actual Results:

The test methods get executed in a random sequence instead of alphabetical order since we are running them in parallel.

The console output shows that the methods have been executed in a random sequence.

Note: The same works fine if we simply remove the parallel and thread-count attribute from our testng.xml file.

One thing important to consider here is that you would only face this problem when the methods have the same priority. The execution order remains exactly as expected when those methods run with different priorities assigned and in parallel.

Now if your test fail you can learn how to capture screenshot of failed tests easily through this video.

You can follow the LambdaTest YouTube Channel by scanning the below QR code and stay up to date with the latest tutorials around Selenium automation testing, cross browser testing, and more.

Conclusion

In this blog on how to set test case priority in TestNG, we learned about the Priority attribute in TestNG and how we can use it while performing Selenium.

We saw how TestNG handles any negative priority, or when all methods have the same priority and when all methods have different priorities. We also came across the limitation of TestNG, which happens when the methods have the same priority assigned, and they are being run in parallel.

I hope you enjoyed reading this Selenium TestNG tutorial on how to set test case priority in TestNG and understood about setting test case priority in TestNG.

Happy Testing !!