Preamble

never in the field of software development have so many owed so much to so few lines of code

about JUnit
— Martin Fowler

…if JUnit is so great then why bother with TestNG?!

Note
My (biased) observations

People are moving from JUnit to TestNG once they start to write integration and end-to-end tests (e.g. with Selenium). Apparently, on unit tests level you can put up with JUnit deficiencies, but once you go outside the unit tests realm, JUnit stands in your way.

frameworki są proste - rozpoznaj metody testowe, wykonaj, opakuj exceptiony w raport frameworki do testów są proste - arrange/act/assert lub given/when/then

Tomek Kaczanowski

img/2013_junit_and_testng/cover_junit_a4_250.jpg img/2013_junit_and_testng/cover_testng_a4_250.jpg

zatrudniamy QA, pisanie książki to osobny temat

Who is Who

img/2013_junit_and_testng/google_trends.png
Google Trends, April 2013
  • JUnit
    • mother of all testing frameworks (at least in Java world)
    • Kent Beck, Erich Gamma
  • TestNG
    • pretendent (or runner-up)
    • Cedric Beust

StackOverflow questions (August 2013):

testNG powstało bo wkurzyły go założenia JUnit założenia: testy są niezależne, core ma być mały; zastuj JUnit przy 3.8.1, TestNG wkracza z annotacjami, grupami, testami parametryzowanymi itp, potem JUnit nadgania, do tego grupa pościgowa - Spock, ScalaTest, BDD za plecami różnica w rozwoju, bezkrólewie vs. plan (chart from Google trends)

Execution Model

  • JUnit: class instantiated before every test method execution
  • TestNG: class instantiated once

pain przy IDE, z drugiej strony w TestNG łatwiej przechować dane między wywołaniami metod testowych

Set-Up

JUnit

TestNG

method

@Before/After

@Before/AfterMethod

class

@Before/AfterClass

@Before/AfterClass

group

-

@Before/AfterGroup

suite

-

@Before/AfterSuite

test [1]

-

@Before/AfterTest

łatwiej wystartować serwer przed grupą testów, w JUnit przez rule

Parameterized Tests - TestNG

pseudocode 
public class MoneyParameterizedTest {

    @DataProvider
    private static final Object[][] getMoney() {
        return new Object[][] {
            new Object[] {10, "USD"},
            new Object[] {20, "EUR"}
        );
    }

    @Test(dataProvider="getMoney")
    public void constructorShouldSetAmountAndCurrency(int amount, String currency) {
        Money money = new Money(amount, currency);

        assertEquals(amount, money.getAmount());
        assertEquals(currency, money.getCurrency());
    }
}

Parameterized Tests - JUnit

  • default JUnit implementation is unusable
  • JUnitParams to the rescue
pseudocode 
@RunWith(JUnitParamsRunner.class)
public class MoneyParameterizedTest {

    private static final Object[] getMoney() {
                return $(
                    $(10, "USD"),
                    $(20, "EUR")
            );
    }

    @Test
    @Parameters(method = "getMoney")
    public void constructorShouldSetAmountAndCurrency(int amount, String currency) {
        Money money = new Money(amount, currency);

        assertEquals(amount, money.getAmount());
        assertEquals(currency, money.getCurrency());
    }
}
  • syntactic sugar
pseudocode 
@Parameters({"10, USD", "20, EUR"})

jak obserwuje testy to widzę że w JUnit się tego mało używa wszystko fajnie ale runnery

Groups of Tests

  • TestNG
pseudocode 
@Test(groups = {"e2e", "inprogress"})
  • JUnit
pseudocode 
public interface FastTests { /* category marker */ }
public interface SlowTests { /* category marker */ }

public class A {
  @Test
  public void a() {
    fail();
  }

  @Category(SlowTests.class)
  @Test
  public void b() {
  }
}

@Category({SlowTests.class, FastTests.class})
public class B {
  @Test
  public void c() {

  }
}

@RunWith(Categories.class)
@IncludeCategory(SlowTests.class)
@SuiteClasses( { A.class, B.class }) // Note that Categories is a kind of Suite
public class SlowTestSuite {
  // Will run A.b and B.c, but not A.a
}

@RunWith(Categories.class)
@IncludeCategory(SlowTests.class)
@ExcludeCategory(FastTests.class)
@SuiteClasses( { A.class, B.class }) // Note that Categories is a kind of Suite
public class SlowTestSuite {
  // Will run A.b, but not A.a or B.c
}

łatwo przerzucać testy między grupami, grupa "in progress"

Concurrency

  • TestNG
pseudocode 
@Test(threadPoolSize = 3, invocationCount = 20)
public void concurrencyTest() {
    System.out.print(" " + Thread.currentThread().getId());
}
  • JUnit - tempus-fugit
pseudocode 
@Rule public ConcurrentRule concurrently = new ConcurrentRule();
@Rule public RepeatingRule repeatedly = new RepeatingRule();

@Test
@Concurrent(count = 7)
@Repeating(repetition = 100)
public void idsShouldBeUnique() {
    System.out.print(" " + Thread.currentThread().getId());
}

Success Percentage

  • TestNG
pseudocode 
@Test(successPercentage = 70)
public void shouldFailRoughlyEveryFourthTime() {
    for (int i = 0; i < 100; i++) {
        assertTrue(Math.random() < 0.75);
    }
}
  • JUnit?
    • probably need to write custom rule

przy zewnętrznych zasobach, przy testach na prawdopodobieństwo

Parameters

  • JUnit - paranoic about parameters passing to test or configuration methods
  • TestNG - allows
pseudocode 
@Parameters("server.url")
@BeforeClass
public void setUp(@Optional("127.0.0.1:8080") String serverUrl) {
    this.serverUrl = serverUrl;
    log.info("expecting server at {}", serverUrl);
}

przekazujesz np. przez Maven Surefire

Dependent Test Methods

  • JUnit says: "this is evil!"
  • TestNG says: "meh…"
pseudocode 
@Test
public void testAddition() {
    // adds user X to the system
    // verifies it exists, by issuing SQL query against the database
}

@Test(dependsOnMethods = "testAddition")
public void testDeletion() {
    // removes user X
    // makes sure it does not exist, by issuing SQL query against the database
}

fail fast, logical dependencies, can be method to method, method to group JUnit - łagodne formy uporządkowania, w TestNG masz IMethod cośtam

Tools

  • Every tool/framework supports JUnit
    • Support for JUnit comes first
    • Many extensions/plugins available
  • All major players (IDEs, Maven, Jenkins, Spring, Arquillian) support TestNG
    • but it happens that the support comes after some time

Pareto rule rules. Pax-exam. Some bonus from Mockito.

Expected Exceptions

Note Does not matter, use Catch-Exception instead
  • TestNG
pseudocode 
@Test(expectedExceptions = PlaneFullException.class)
public void shouldNotAllowToOverbookPlane() {
    plane.bookAllSeats();
    plane.bookOneMoreSeat();
}
  • JUnit
    • the same + ExpectedException rule

Assertions

Note Does not matter, use FEST or Hamcrest instead
  • JUnit: assertEquals(expected, actual)
  • TestNG: assertEquals(actual, expected)

Cedric kombinuje z asercjami http://beust.com/weblog/2012/07/29/reinventing-assertions/

The End

Great things are done by a series of small things brought together.

— Vincent Van Gogh

It is many small things which sum up and give you a much better experience.

JUnit (+ friends: Tempus-Fugit, JUnitParams) is almost as poweful as TestNG.
Almost, because some concepts are done wrong (groups, parameterized tests, test dependencies).

Links