If you want to evaluate a testing plan, you must also consider your alerting plan.
Alerting and testing are complimentary, both serve to identify defects. Testing typically serves to identify defects before code is deployed to production, while alerting typically notifies developers of an issue with a running system.
You need to consider both testing and alerting to create an effective defect mitigation plan.
Start by asking yourself a few important questions:
What is the businesses’ tolerance for defects in production of this system?
Can we easily rectify production issues with this system postmortem (after being alerted), or will it cause non-trivial damage to business operations and reputation?
Are developers capable and willing to do on-call fixes to production systems? How much ongoing cost is there in training?
Once you’ve identified your tolerance for defects in production (and ability to fix them), you can better evaluate what preventative measures should live as real-time alerting, and what measures should live as pre-deployment tests.
Often, I find that using alerting to catch errors is a magnitude less of a time investment compared to developing comprehensive integration testing to catch the same defects. The downside is that resolving the alerts still requires developer time and manual effort.
In my opinion, alerting on production systems is more fundamental than automated testing, but both should play some role in designing a defect mitigation plan.
The point is: You can’t design a good testing plan without having an alerting plan.
One very peculiar issue I’ve run into during development is an issue where I have a parent pom.xml file and a child pom.xml file. The child pom.xml file imports the parent artifact as a dependency.
Now, when we make changes to dependencies in the parent pom.xml file, we don’t see them reflected when building the child pom.xml file, the child is still pulling in an unwanted dependency version from the parent.
What’s the issue?
Is the parent artifact being published to the local maven repository correctly? Yes it is, that’s not the issue.
A common problem for Java developers that wish to get comprehensive unit test coverage is handling the mocking of singletons that are implemented using static method calls.
Let’s look at how we can mock singleton behavior using three common Java testing libraries Mockito, EasyMock and JMockit.
Part 1: Write Code to Test
To start with, we need a simple contrived example. In this example, we have a class ClassUnderTest that accesses a singleton Singleton and then, based on the return value, calls a method in DummyApiClient.
The issue is, we need to be able to mock the behavior of the static instance of Singleton in order to test ClassUnderTest.
public class ClassUnderTest { private DummyApiClient dummyApiClient; public ClassUnderTest(DummyApiClient dummyApiClient){ this.dummyApiClient = dummyApiClient; } public void methodUnderTest(){ if(Singleton.getSingleton().getBool()){ dummyApiClient.doPositive(); } else { dummyApiClient.doNegative(); } }}
import java.util.Random;public class Singleton { static Singleton singleton; static { singleton = new Singleton(); } public static Singleton getSingleton(){ return singleton; } public Boolean getBool(){ return new Random().nextBoolean(); }}
public class DummyApiClient { public void doPositive(){ System.out.println("positive");}; public void doNegative(){ System.out.println("negative");};}
Wow! It’s now possible to mock static methods with mockito, without the additional dependency of PowerMock! Since version 3.4 of Mockito (PR), we can mock static methods using the mockStatic command. (examples)
First, let’s add the required dependencies to our pom.xml file. We need to use JUnit and mockito-inline (regular mockito-core will not work).
As of the time of writing, EasyMock does not support mocking of static methods without the use of an additional library; In our case, we will use PowerMock to support mocking static methods.
First, let’s update our pom.xml to reflect the libraries we are using:
import org.junit.Before;import org.junit.Test;import org.junit.runner.RunWith;import org.powermock.api.easymock.PowerMock;import org.powermock.core.classloader.annotations.PrepareForTest;import org.powermock.modules.junit4.PowerMockRunner;import static org.easymock.EasyMock.expect;import static org.powermock.api.easymock.PowerMock.mockStatic;@RunWith(PowerMockRunner.class)@PrepareForTest(Singleton.class)public class ClassUnderTestTest { DummyApiClient dummyApiClient; Singleton singletonMock; ClassUnderTest unit; @Before public void beforeEach(){ dummyApiClient = PowerMock.createMock(DummyApiClient.class); singletonMock = PowerMock.createMock(Singleton.class); mockStatic(Singleton.class); expect(Singleton.getSingleton()).andReturn(singletonMock); unit = new ClassUnderTest(dummyApiClient); } @Test public void testMethodUnderTestPositive(){ expect(singletonMock.getBool()).andReturn(true); dummyApiClient.doPositive(); PowerMock.expectLastCall(); PowerMock.replayAll(); unit.methodUnderTest(); } @Test public void testMethodUnderTestNegative(){ expect(singletonMock.getBool()).andReturn(false); dummyApiClient.doNegative(); PowerMock.expectLastCall(); PowerMock.replayAll(); unit.methodUnderTest(); }}
Resetting Mocks: Note how mockStatic(Singleton.class); doesn’t have a corresponding reset call. This is because PowerMock resets mocks using the @PrepareForTest annoation. (stack overflow post)