Tag Archives: junit

Unexpected Pytest things for JUnit Developers

Posted on by
Reply

I used PyTest for the first time recently. It’s similar to JUnit as one would expect (since they are both XUnit frameworks.) There were a few things that were unexpected though so writing a a blog post. That’ll help me remember if I forget 🙂 Plus posting something on the internet is a great way to find out what you misunderstood if anything!

Getting started

I redownloaded Python as it is been a couple years since I last used Python on my machine and I don’t remember the current state of affairs. Then I created a virtual env for this blog post. 

python3 -m venv blog-post
cd blog-post
source bin/activate

Next I installed pytest and pytest-mock inside the virtual environment. (adding it to the requirements file isn’t enough. I think this since the IDE needs them to run the tests)

pip3 install pytest
pip3 install pytest-mock

I confirmed my VS Code had the Microsoft Python extension configured. It did. I also set up my project for Pytest integration as described in Eric’s blog post so I can run tests in my IDE.

Surprise #1: unittest vs pytest

Python has a built in testing framework called unittest and an add on one called pytest. Java has JUnit and TestNG so it certainly isn’t unusual to have more than one option. I was surprised one was built in though. And this difference was definitely one I had to pay attention to when looking at docs. More on the differences if curious.

Creating hello world test

I created this directory structure:

math.py contains:

def add(x, y):
    return x + y

And test_add.py contains:

import pytest

import src.my_mod.math as target

def test_add():
    actual = target.add(4, 5)
    assert 9 == actual

I had to remember to use the flask icon to run in the IDE, but I’m not going to call that a surprise.

Surprise #2: Running at the command line

Running at the command line definitely yielded a surprise. This does not work because of the paths. (pytest uses a different path when the command vs a module

pytest
pytest tests
pytest tests/test_add.py
python -m pytest

Either of the following work. I like the first one because I don’t have to change the name of the file I want to run. (Although I’m using the IDE more anyway.)

python -m pytest tests
python -m pytest tests/test_add.py

Surprise #3: Assertion messages

The basics of assertions make sense to me. assert False, assert True, assert x == y. So far so good.

Failing assertions are good as well. Having an incorrect expected value gives output like the following. It shows the expanded version of actual. And it shows those final expected vs actual in the short test summary.

def test_add():
        actual = target.add(4, 5)
>       assert 7 == actual
E       assert 7 == 9

tests/test_add.py:7: AssertionError
=========================== short test summary info ============================
FAILED tests/test_add.py::test_add - assert 7 == 9
============================== 1 failed in 0.01s ===============================
Finished running tests!

In JUnit, it is good practice to add an assertion message to get more details. The expanded values are still there. However, the short test summary info only shows your custom message. I’m not using the message as I prefer to have the summary messages included what was expected vs actual and I don’t want to have to repeat the code to make that so.

    def test_add():
        actual = target.add(4, 5)
>       assert 7 == actual, "addition result incorrect"
E       AssertionError: addition result incorrect
E       assert 7 == 9

tests/test_add.py:7: AssertionError
=========================== short test summary info ============================
FAILED tests/test_add.py::test_add - AssertionError: addition result incorrect
============================== 1 failed in 0.02s ===============================
Finished running tests!

Surprise #4: Logging output

When not doing TDD (ex: testing code that already exists), I like to write a test that prints the expected value and then adding an assertion to match. I also do this when applying the golden master pattern (declare what exists to be working and codify it)

def test_add():
    actual = target.add(4, 5)
    print("Jeanne debug output " + str(actual))

I was baffled why there was no output. Reason? The print’s are only printed if the test contains a failing assertion!

On one of my machines, the prints don’t output in VS Code but do at the command line. On my other machine, it prints in both. It might be settings related, but not sure.

Surprise #5: Conventions matter

I accidentally created a file in tests that didn’t begin with test. I learned this is critical and my test got ignored until I renamed the file. (Yes, I’d have known this if I had read docs)

Surprise #6: When you made a syntax error…

If you make a certain syntax error, all the tests fail. This is scary until you realize what’s going on.

On to mocking

Here’s a simple test method to mock

import requests

def get_url(url):
    return requests.get(url)

And the test:

def test_get_url(mocker):
    url = 'https://python.org'
    data = 'html data from url'
    mocker.patch('requests.get', return_value=data)

    actual = target.get_url(url)
    assert data == actual
    requests.get.assert_called_once_with(url)

I like that the return_value is specified on the same line as the mock call. The assert for the parameter is at the end which differs from Java. I also like that there isn’t an elaborate dependency injection system. For more on mocking see Eric’s write up.

Surprise #7: My mock is a string???

When I wrote this line of code in error, I got an error that the ‘str’ object is not callable.

mocker.patch('requests.get', data)

In hindsight, this makes sense. I replace the “get” method with the string value data rather than setting data as a return value.

Surprise #8: operating system environment variables

I was mocking out os.env by setting a dictionary. I thought I was setting the environment variables to empty

mock.patch.dict(os.environ, {})

However, when I ran this on a CI server, it failed because the OS variable was in fact set. I learned that the dictionary is added to the existing variables by default. The fix is

mock.patch.dict(os.environ, {}, clear=True)

JavaOne: JUnit 5 Features, Architecture and Extensibility

Posted on by
Reply

“JUnit 5 Features, Architecture and Extensibility”

Speaker: Steve Moyer, Stephen Seltzer & Niraja Ramesh
Deck on GitHub

For more blog posts from JavaOne, see the table of contents

This is “part 2” because last year they gave a session. It’s an expanded version of that talk now that JUnit 5 is done.

Rationale

  • Decouple test execution/reporting from test definition/provisioning
  • JUnit 8 features. ex: message nw last param and can be lambda
  • migration-support – JUnit 4 rules
  • junit-platform-console – to execute from command line; have to build full classpath

Principles

  • Prefer extension points over features
  • An extension point shoul be good at one thing
  • Should be hard to write tests that behave differently based on how run
  • Test should be easy to understand
  • Minimize dependencies; epecially third party

Features

  • Meta Annotations – compose annotations out of other annotations
  • Tags/filters
  • Dependency injection through params on constructors/methods
  • Default interface methods

Code examples (also on GitHub

  • Assertions – assertEquals(expected, actual, message)
  • Skip tests @Disabled
  • Lambdas – assertTrue(a, () -> message)
  • Grouped assertions – assertAll(message, () -> assertEquals(expected, actual, essage))
  • Exceptions – assertThrows
  • Assumptions assumeTrue, assumeFalse, assumingThat – he last one sets a scope (lambda) for the assumption and continues after that block regardless
  • Meta-annotations Create annotation with other annotation. Similarly can tag tests with annotations. Eclipse lets you specify filters in run config
  • Repeated tests – pass how many times to run along with what to display – can pass current repetition an tota reptitions to display name. Not for performance testing (other tools for that) but good for idempotence testing
  • Parameterized tests – can feed in CSVFileSource, CSVSource (CSV as strings), MethodSource. See docs for implicit conversions [didn’t mention value source :(]
  • Test templates – designed to be invoked multiple times and run like indepenendent test method. Parameterized tests are an implmentation of test templates
  • Lots of extension points and callbacks. Can programmatially decide whether to run tests.
  • ParameterResolvers – TestInfo (has display name, test class and test method), TestReporter (allows publising to output report) – can have JUnit pass as param to test
  • Callback order – BeforeAll, BeforeEach, BeforeTestExecution, Test, AfterTestExcution, AfterEach, AfterAll. Does parent class before child on Before*** and after child on After***
  • Dynamic tests – use @TestFactory – have display name and Executable interface
  • Grouped tests – Can use traditional for loop, assertAll or factory tests. Different output. Traditional for loop only ells you about first failure. With assertAll, get all failures but one faling test. With factory tests, you have a failing test for each faiure.

Future