A TestNG reporter that uploads the results to a ReportPortal server.

We use Google Analytics for sending anonymous usage information as agent's name andversion and the java client's name and version after the successful launch start. This information might help us to improve both ReportPortal backend and client sides. Used by the ReportPortal team only and not for sharing with 3rd parties.

• Objects interrelation TestNG - ReportPortal
• Dependencies
• Install listener
  • Listener class
  • Listener parameters
  • Maven Surefire plugin
  • Specify listener in testng.xml
  • Custom runner
  • Using command line
  • Using @Listeners annotation
  • Using ServiceLoader
• Code example How to overload params in run-time
• Example and step-by-step instruction with logback
• Example and step-by-step instruction with Log4j

TestNG provides support for attaching custom listeners, reporters, annotation transformers and method interceptors to your tests. Handling events

TestNG agent can handle next events:

• Start launch
• Finish launch
• Start suite
• Finish suite
• Start test
• Finish test
• Start test step
• Successful finish of test step
• Fail of test step
• Skip of test step
• Start configuration (All «before» and «after» methods)
• Fail of configuration
• Successful finish of configuration
• Skip configuration

TestItem – report portal specified object for representing: suite, test, method objects in different test systems. Used as tree structure and can be recursively placed inside himself.

Minimum supported TestNG version: 7.1.0

Add to POM.xml

dependency

<repositories>
     <repository>
        <snapshots>
          <enabled>false</enabled>
        </snapshots>
        <id>bintray-epam-reportportal</id>
        <name>bintray</name>
        <url>http://dl.bintray.com/epam/reportportal</url>
     </repository>
</repositories>

<dependency>
  <groupId>com.epam.reportportal</groupId>
  <artifactId>agent-java-testng</artifactId>
  <version>5.0.8</version>
</dependency>
<!-- TODO Leave only one dependency, depends on what logger you use: -->
<dependency>
  <groupId>com.epam.reportportal</groupId>
  <artifactId>logger-java-logback</artifactId>
  <version>5.0.3</version>
</dependency>
<dependency>
  <groupId>com.epam.reportportal</groupId>
  <artifactId>logger-java-log4j</artifactId>
  <version>5.0.3</version>
</dependency>

Download package here. Choose latest version.

By default, TestNG attaches a few basic listeners to generate HTML and XML reports. For reporting TestNG test events (ie start of test, successful finish of test, test fail) to ReportPortal user should add ReportPortal TestNg listener to run and configure input parameters.

Description of listeners input parameters and how to configure it see “Parameters” in Configuration section. Which are common for all JVM based agents.

com.epam.reportportal.testng.ReportPortalTestNGListener

There are several ways how to install listener:

• Maven Surefire plugin
• Specify listener in testng.xml
• Custom runner
• Using command line
• Using @Listeners annotation
• Using ServiceLoader

Please note, that listener must be configured in a single place only. Configuring multiple listeners will lead to incorrect application behavior.

Maven surefire plugin allows configuring multiple custom listeners. For logging to Report Portal user should add Report Portal listener to “Listener” property.

<plugins>
    [...]
      <plugin>
        <groupId>org.apache.maven.plugins</groupId>
        <artifactId>maven-surefire-plugin</artifactId>
        <version>2.15</version>
        <configuration>
          <properties>
            <property>
              <name>usedefaultlisteners</name>
              <value>false</value> <!-- disabling default listeners is optional -->
            </property>
            <property>
              <name>listener</name>
              <value>com.epam.reportportal.testng.ReportPortalTestNGListener</value>
            </property>
          </properties>
        </configuration>
      </plugin>
    [...]
</plugins>

If you use maven surefire plugin set report portal listener only in "listener" property in pom.xml.

Here is how you can define report portal listener in your testng.xml file.

<suite>
   
  <listeners>
    <listener class-name="com.example.MyListener" />
    <listener class-name="com.epam.reportportal.testng.ReportPortalTestNGListener" />
  </listeners>
.....

TestNG allows users to create own runners. In this case user can instantiate listener by himself and add it to TestNG object.

ReportPortalTestNGListener listener = new ReportPortalTestNGListener();
TestNG testNg = new TestNG();
List<String> lSuites = Lists.newArrayList();
Collections.addAll(lSuites, “testng_suite.xml”);
testNg.setTestSuites(lSuites);
testNg.addListener((Object) listener);
testNg.run();

Assuming that you have TestNG and Report Portal client jars in your class path, you can run TestNG tests with Report Portal listener as follows:

java org.testng.TestNG testng1.xml –listener com.epam.reportportal.testng.ReportPortalTestNGListener

Report Portal listener can be set using @Listener annotation.

@Listeners({ReportPortalTestNGListener.class})
public class FailedParameterizedTest {
…..
}

@Listeners annotation will apply to your entire suite file, just as if you had specified it in a testng.xml file. Please do not restrict its scope.

JDK offers a very elegant mechanism to specify implementations of interfaces on the class path via the ServiceLoader class. With ServiceLoader, all you need to do is create a jar file that contains your listener(s) and a few configuration files, put that jar file on the classpath when you run TestNG and TestNG will automatically find them. You can use service loaded mechanism for adding Report Portal listener to test execution. Please see detailed information here:

http://testng.org/doc/documentation-main.html#testng-listeners 5.17.2 - Specifying listeners with ServiceLoader.

As a sample you can use code for Override UUID in run-time

	public static class MyListener extends BaseTestNGListener {
		public MyListener() {
			super(new ParamOverrideTestNgService());
		}
	}

	public static class ParamOverrideTestNgService extends TestNGService {
		public ParamOverrideTestNgService() {
			super(getLaunchOverriddenProperties());
		}

		private static Supplier<Launch> getLaunchOverriddenProperties() {
			ListenerParameters parameters = new ListenerParameters(PropertiesLoader.load());
			parameters.setApiKey("my crazy uuid");
			ReportPortal reportPortal = ReportPortal.builder().withParameters(parameters).build();
			StartLaunchRQ rq = buildStartLaunch(reportPortal.getParameters());
			return new Supplier<Launch>() {
				@Override
				public Launch get() {
					return reportPortal.newLaunch(rq);
				}
			};
		}

		private static StartLaunchRQ buildStartLaunch(ListenerParameters parameters) {
			StartLaunchRQ rq = new StartLaunchRQ();
			rq.setName(parameters.getLaunchName());
			rq.setStartTime(Calendar.getInstance().getTime());
			rq.setAttributes(parameters.getAttributes());
			rq.setMode(parameters.getLaunchRunningMode());
			if (!Strings.isNullOrEmpty(parameters.getDescription())) {
				rq.setDescription(parameters.getDescription());
			}

			return rq;
		}
	}

There are two modules under Example project which represent agent usage with Lo4j and Logback loggers:

• https://github.com/reportportal/examples-java/tree/master/example-testng-log4j
• https://github.com/reportportal/examples-java/tree/master/example-testng-logback