JUnit vs TestNG Difference - Features, Assertions and Annotiations

Publication : 30 août 2020

This is an introduction into the differences regarding the features, the assert commands ("assertFalse", "assertEquals", etc.) and the annotations ("@BeforeEach", etc.) of the two popular testing frameworks "JUnit" and "TestNg".

The decision for the suitable testing framework is important to ensure the quality of your software. The best testing framework can differ regarding the size and type of your software and its dependencies.
The JUnit version 5 and TestNG version 7 are used here.

Features

Both frameworks have parameterized tests, timeout feature and exception tests. But TestNG has some more advanced features and it is also possible to do more than unit testing.
These two testing frameworks differ in the following features and functionality:

JUnit

Installed by default in your IDE
Configuration done only from Java test classes
No need for an additional XML configuration file
Bigger community
More documentation
Adjusted for unit testing

TestNG

Test suite configured through the testng.xml file
Create groups of tests that belong to different classes through the "testng.xml" file
Dependency tests (command: "dependsOnMethod")
Parallel test execution (attribute in "testng" XML file: "parallel")
Configure the order in which test cases are executed (command: "priority")
HTML report is created by default

TestNG does organize your unit test a little bit different, than JUnit. All test cases that you created belong always to a test suite, which are defined in the XML file "testng".
An example of the XML file "testng":

<suite name = "MySuite">

   <test name = "testAddObject">
      <classes>
         <class name = "Test1" />
      </classes>
   </test>
  
   <test name = "testDeleteObject">
      <classes>
         <class name = "Test2" />
      </classes>
   </test>
  
</suite>

Assertions

These two frameworks have the same assertion methods ("assertEquals", "assertFalse", "assertAll", "assertThrows", etc.). But they differ in the order of the method arguments, which look like this:

AssertEquals

"assertEquals" is available for all data types. It does check, if the passed object values are the same. This method uses "equals()" to compare the two passed objects.

JUnit:

assertEquals(Object expected, Object actual, String message)

TestNG:

assertEquals(Object actual, Object expected, String message)

This difference is the same for all other assertions. Example: "assertTrue", "assertSame", etc.

These are the assertions that are unique to a certain testing framework:

AssertTimeout

This assertion checks if the execution time did not exceeded the defined time value.

JUnit:

assertTimeout(Duration timeout, Executable executable)

You have to pass the maximum time (as a "Duration" object) that the function in the second argument should take. Example:

@Test
public void testMultiplicationDuration() {
	assertTimeout(Duration.ofSeconds(5), calculation.multiplication(545455, 2545455));
}

TestNG:

There is no assertion for this. But you can do the same by adding the command "timeout" to the annotation "@Test". The time is in milliseconds.

@Test(timeOut = 5000)
public void testCalculation() throws InterruptedException {
	Assert.assertEquals(calculation.multiplication(545455, 2545455), 1388431157025);
}

assertEqualsDeep

This is available only in TestNG. This does compare the elements of a "Map" or "Set".

assertEqualsDeep(java.util.Map<?,?> actual, java.util.Map<?,?> expected, java.lang.String message)

assertEqualsDeep(java.util.Set<?> actual, java.util.Set<?> expected, java.lang.String message)

If you want to do this in JUnit, then you can do it directly with "assertTrue" like this:

assertTrue(expectedMap.equals(actualMap));

Annotations

Both frameworks have nearly the same type of annotations. They differ in in their name.

@BeforeEach - @AfterEach

In TestNG these annotations are called "@BeforeMethod" and "@AfterMethod". This runs a method every time before or after a test method (method with annotation "@Test", "@RepeatedTest", "@ParameterizedTest", or "@TestFactory").
There also annotations that are not available in JUnit such as "@DataProvider".

@RepeatedTest

This is only available in JUnit and it does repeat a test method a certain number of times. Example:

@RepeatedTest(10)
public void test10Times() {
	System.out.println("Repeated test - Run 10 times");
}

You can do this in a different way in TestNG:

@Test(invocationCount = 10)
public void test10Times() {
	System.out.println("Repeated test - Run 10 times");
}

@DataProvider

A "dataProvider" allows to pass parameterized data to your method arguments.
This is available in testNG and it is not available in JUnit by default. If you want to use this in JUnit, then you have to add the dependency "JUnit DataProvider" to your project.

The data that you want to pass to your method is created in a separate method (with the annotation "@DataProvider" which will be your "data provider" and it return your created data as an Object array.
You can then pass your created data through your method arguments by using the annotation attribute "dataProvider".

Example:

@DataProvider
public Object[][] getTenCostCategoryItems() {
    Object[][] costCategoryItems = new Object[][]{{"Tire Change", 2000, 1}, {"Paint Color", 6000, 2}, {"Repair Engine", 8000, 3}, {"Repair Windshield", 7000, 4}, {"Change Entertainment System", 4600, 5}, {"Repair Audio Boxes", 8000, 6}, {"Change Seating", 7600, 7}, {"Change Oil", 9000, 8}, {"Repair Door", 8000, 9}, {"Change Lights", 4000, 10}};
    return costCategoryItems;
}

@Test(dataProvider = "getTenCostCategoryItems")
public void testAddTenAndCheck(String costCategoryItemTitle, Integer costCategoryItemPriceCent, Integer index) {
	CostCategory expectedCostCategoryItem = new CostCategory(costCategoryItemTitle, costCategoryItemPriceCent);
    this.costCategoryService.add(expectedCostCategoryItem);
    CostCategory actualCostCategoryItem = this.costCategoryService.getById(index, classObject);
    try {
        assertEquals(actualCostCategoryItem.getTitle(), expectedCostCategoryItem.getTitle(), "Added title should be saved in database.");
        assertEquals(actualCostCategoryItem.getCentPrice(), expectedCostCategoryItem.getCentPrice(), "Added CentPrice should be saved in database.");
        assertEquals(actualCostCategoryItem.getId(), index, "PK must be " + index + ".");
    } catch (NullPointerException e) {
        fail("Created object was not saved in database");
    }
}

In this example the test method "testAddTenAndCheck" will get ten times a String "costCategoryItemTitle", an Integer "costCategoryItemPriceCent" and an Integer "index" from the Array "costCategoryItems". These values are available through the arguments of this test method.

This was an introduction to the differences between JUnit and TestNG. There are several other assertions and annotations as well. Please refer to the documentation for more information.

Dependency "JUnit DataProvider":
https://mvnrepository.com/artifact/com.tngtech.java/junit-dataprovider

TestNG Documentation:
https://www.javadoc.io/doc/org.testng/testng/7.0.0/org/testng/Assert.html

JUnit Documentation:
https://junit.org/junit5/docs/5.0.1/api/org/junit/jupiter/api/Assertions.html