There are some key challenges around microservices architecture testing that we are facing. The selection of right tools is one of that elements that helps us deal with the issues related to those challenges. First, let’s identify the most important elements involved into the process of microservices testing. These are some of them:
- Teams coordination – with many independent teams managing their own microservices, it becomes very challenging to coordinate the overall process of software development and testing
- Complexity – there are many microservices that communicate to each other. We need to ensure that every one of them is working properly and is resistant to the slow responses or failures from other microservices
- Performance – since there are many independent services it is important to test the whole architecture under traffic close to the production
Let’s discuss some interesting frameworks helping that may help you in testing microservice-based architecture.
Components tests with Hoverfly
Hoverfly simulation mode may be especially useful for building component tests. During component tests we are verifying the whole microservice without communication over network with other microservices or external datastores. The following picture shows how such a test is performed for our sample microservice.
Hoverfly provides simple DSL for creating simulations, and a JUnit integration for using it within JUnit tests. It may orchestrated via JUnit @Rule. We are simulating two services and then overriding Ribbon properties to resolve address of these services by client name. We should also disable communication with Eureka discovery by disabling registration after application boot or fetching list of services for Ribbon client. Hoverfly simulates responses for PUT and GET methods exposed by passenger-management and driver-management microservices. Controller is the main component that implements business logic in our application. It store data using in-memory repository component and communicates with other microservices through @FeignClient interfaces. By testing three methods implemented by the controller we are testing the whole business logic implemented inside trip-management service.
@SpringBootTest(properties = {
"eureka.client.enabled=false",
"ribbon.eureka.enable=false",
"passenger-management.ribbon.listOfServers=passenger-management",
"driver-management.ribbon.listOfServers=driver-management"
})
@RunWith(SpringRunner.class)
@AutoConfigureMockMvc
@FixMethodOrder(MethodSorters.NAME_ASCENDING)
public class TripComponentTests {
ObjectMapper mapper = new ObjectMapper();
@Autowired
MockMvc mockMvc;
@ClassRule
public static HoverflyRule rule = HoverflyRule.inSimulationMode(SimulationSource.dsl(
HoverflyDsl.service("passenger-management:80")
.get(HoverflyMatchers.startsWith("/passengers/login/"))
.willReturn(ResponseCreators.success(HttpBodyConverter.jsonWithSingleQuotes("{'id':1,'name':'John Walker'}")))
.put(HoverflyMatchers.startsWith("/passengers")).anyBody()
.willReturn(ResponseCreators.success(HttpBodyConverter.jsonWithSingleQuotes("{'id':1,'name':'John Walker'}"))),
HoverflyDsl.service("driver-management:80")
.get(HoverflyMatchers.startsWith("/drivers/"))
.willReturn(ResponseCreators.success(HttpBodyConverter.jsonWithSingleQuotes("{'id':1,'name':'David Smith','currentLocationX': 15,'currentLocationY':25}")))
.put(HoverflyMatchers.startsWith("/drivers")).anyBody()
.willReturn(ResponseCreators.success(HttpBodyConverter.jsonWithSingleQuotes("{'id':1,'name':'David Smith','currentLocationX': 15,'currentLocationY':25}")))
)).printSimulationData();
@Test
public void test1CreateNewTrip() throws Exception {
TripInput ti = new TripInput("test", 10, 20, "walker");
mockMvc.perform(MockMvcRequestBuilders.post("/trips")
.contentType(MediaType.APPLICATION_JSON_UTF8)
.content(mapper.writeValueAsString(ti)))
.andExpect(MockMvcResultMatchers.status().isOk())
.andExpect(MockMvcResultMatchers.jsonPath("$.id", Matchers.any(Integer.class)))
.andExpect(MockMvcResultMatchers.jsonPath("$.status", Matchers.is("NEW")))
.andExpect(MockMvcResultMatchers.jsonPath("$.driverId", Matchers.any(Integer.class)));
}
@Test
public void test2CancelTrip() throws Exception {
mockMvc.perform(MockMvcRequestBuilders.put("/trips/cancel/1")
.contentType(MediaType.APPLICATION_JSON_UTF8)
.content(mapper.writeValueAsString(new Trip())))
.andExpect(MockMvcResultMatchers.status().isOk())
.andExpect(MockMvcResultMatchers.jsonPath("$.id", Matchers.any(Integer.class)))
.andExpect(MockMvcResultMatchers.jsonPath("$.status", Matchers.is("IN_PROGRESS")))
.andExpect(MockMvcResultMatchers.jsonPath("$.driverId", Matchers.any(Integer.class)));
}
@Test
public void test3PayTrip() throws Exception {
mockMvc.perform(MockMvcRequestBuilders.put("/trips/payment/1")
.contentType(MediaType.APPLICATION_JSON_UTF8)
.content(mapper.writeValueAsString(new Trip())))
.andExpect(MockMvcResultMatchers.status().isOk())
.andExpect(MockMvcResultMatchers.jsonPath("$.id", Matchers.any(Integer.class)))
.andExpect(MockMvcResultMatchers.jsonPath("$.status", Matchers.is("PAYED")));
}
}
The tests visible above verify only positive scenarios. What about testing some unexpected behaviour like network delays or server errors? With Hoverfly we can easily simulate such a behaviour and define some negative scenarios. In the following fragment of code I have defined three scenarios. In the first of them target service has been delayed 2 seconds. In order to simulate timeout on the client side I had to change default readTimeout for Ribbon load balancer and then disabled Hystrix circuit breaker for Feign client. The second test simulates HTTP 500 response status from passenger-management service. The last scenario assumes empty response from method responsible for searching the nearest driver.
@SpringBootTest(properties = {
"eureka.client.enabled=false",
"ribbon.eureka.enable=false",
"passenger-management.ribbon.listOfServers=passenger-management",
"driver-management.ribbon.listOfServers=driver-management",
"feign.hystrix.enabled=false",
"ribbon.ReadTimeout=500"
})
@RunWith(SpringRunner.class)
@AutoConfigureMockMvc
public class TripNegativeComponentTests {
private ObjectMapper mapper = new ObjectMapper();
@Autowired
private MockMvc mockMvc;
@ClassRule
public static HoverflyRule rule = HoverflyRule.inSimulationMode(SimulationSource.dsl(
HoverflyDsl.service("passenger-management:80")
.get("/passengers/login/test1")
.willReturn(ResponseCreators.success(HttpBodyConverter.jsonWithSingleQuotes("{'id':1,'name':'John Smith'}")).withDelay(2000, TimeUnit.MILLISECONDS))
.get("/passengers/login/test2")
.willReturn(ResponseCreators.success(HttpBodyConverter.jsonWithSingleQuotes("{'id':1,'name':'John Smith'}")))
.get("/passengers/login/test3")
.willReturn(ResponseCreators.serverError()),
HoverflyDsl.service("driver-management:80")
.get(HoverflyMatchers.startsWith("/drivers/"))
.willReturn(ResponseCreators.success().body("{}"))
));
@Test
public void testCreateTripWithTimeout() throws Exception {
mockMvc.perform(MockMvcRequestBuilders.post("/trips").contentType(MediaType.APPLICATION_JSON).content(mapper.writeValueAsString(new TripInput("test", 15, 25, "test1"))))
.andExpect(MockMvcResultMatchers.status().isOk())
.andExpect(MockMvcResultMatchers.jsonPath("$.id", Matchers.nullValue()))
.andExpect(MockMvcResultMatchers.jsonPath("$.status", Matchers.is("REJECTED")));
}
@Test
public void testCreateTripWithError() throws Exception {
mockMvc.perform(MockMvcRequestBuilders.post("/trips").contentType(MediaType.APPLICATION_JSON).content(mapper.writeValueAsString(new TripInput("test", 15, 25, "test3"))))
.andExpect(MockMvcResultMatchers.status().isOk())
.andExpect(MockMvcResultMatchers.jsonPath("$.id", Matchers.nullValue()))
.andExpect(MockMvcResultMatchers.jsonPath("$.status", Matchers.is("REJECTED")));
}
@Test
public void testCreateTripWithNoDrivers() throws Exception {
mockMvc.perform(MockMvcRequestBuilders.post("/trips").contentType(MediaType.APPLICATION_JSON).content(mapper.writeValueAsString(new TripInput("test", 15, 25, "test2"))))
.andExpect(MockMvcResultMatchers.status().isOk())
.andExpect(MockMvcResultMatchers.jsonPath("$.id", Matchers.nullValue()))
.andExpect(MockMvcResultMatchers.jsonPath("$.status", Matchers.is("REJECTED")));
}
}
All the timeouts and errors in communication with external microservices are handled by the bean annotated with @ControllerAdvice. In such cases trip-management microservice should not return server error response, but 200 OK with JSON response containing field status equals to REJECTED.
@ControllerAdvice
public class TripControllerErrorHandler extends ResponseEntityExceptionHandler {
@ExceptionHandler({RetryableException.class, FeignException.class})
protected ResponseEntity handleFeignTimeout(RuntimeException ex, WebRequest request) {
Trip trip = new Trip();
trip.setStatus(TripStatus.REJECTED);
return handleExceptionInternal(ex, trip, null, HttpStatus.OK, request);
}
}
Contract tests with Pact
The next type of test strategy usually implemented for microservices-based architecture is consumer-driven contract testing. In fact, there are some tools especially dedicated for such type of tests. One of them is Pact. Contract testing is a way to ensure that services can communicate with each other without implementing integration tests. A contract is signed between two sides of communication: consumer and provider. Pact assumes that contract code is generated and published on the consumer side, and than verified by the provider.
Pact provides tool that can store and share the contracts between consumers and providers. It is called Pact Broker. It exposes a simple RESTful API for publishing and retrieving pacts, and embedded web dashboard for navigating the API. We can easily run Pact Broker on the local machine using its Docker image.
We will begin from running Pact Broker. Pact Broker requires running instance of postgresql, so first we have to launch it using Docker image, and then link our broker container with that container.
docker run -d --name postgres -p 5432:5432 -e POSTGRES_USER=oauth -e POSTGRES_PASSWORD=oauth123 -e POSTGRES_DB=oauth postgres
docker run -d --name pact-broker --link postgres:postgres -e PACT_BROKER_DATABASE_USERNAME=oauth -e PACT_BROKER_DATABASE_PASSWORD=oauth123 -e PACT_BROKER_DATABASE_HOST=postgres -e PACT_BROKER_DATABASE_NAME=oauth -p 9080:80 dius/pact-broker
The next step is to implement contract tests on the consumer side. We will use JVM implementation of Pact library for that. It provides PactProviderRuleMk2 object responsible for creating stubs of the provider service. We should annotate it with JUnit @Rule. Ribbon will forward all requests to passenger-management to the stub address – in that case localhost:8180. Pact JVM supports annotations and provides DSL for building test scenarios. Test method responsible for generating contract data should be annotated with @Pact. It is important to set fields state and provider, because then generated contract would be verified on the provider side using these names. Generated pacts are verified inside the same test class by the methods annotated with @PactVerification. Field fragment points to the name of the method responsible for generating pact inside the same test class. The contract is tested using PassengerManagementClient @FeignClient.
@RunWith(SpringRunner.class)
@SpringBootTest(properties = {
"driver-management.ribbon.listOfServers=localhost:8190",
"passenger-management.ribbon.listOfServers=localhost:8180",
"ribbon.eureka.enabled=false",
"eureka.client.enabled=false",
"ribbon.ReadTimeout=5000"
})
public class PassengerManagementContractTests {
@Rule
public PactProviderRuleMk2 stubProvider = new PactProviderRuleMk2("passengerManagementProvider", "localhost", 8180, this);
@Autowired
private PassengerManagementClient passengerManagementClient;
@Pact(state = "get-passenger", provider = "passengerManagementProvider", consumer = "passengerManagementClient")
public RequestResponsePact callGetPassenger(PactDslWithProvider builder) {
DslPart body = new PactDslJsonBody().integerType("id").stringType("name").numberType("balance").close();
return builder.given("get-passenger").uponReceiving("test-get-passenger")
.path("/passengers/login/test").method("GET").willRespondWith().status(200).body(body).toPact();
}
@Pact(state = "update-passenger", provider = "passengerManagementProvider", consumer = "passengerManagementClient")
public RequestResponsePact callUpdatePassenger(PactDslWithProvider builder) {
return builder.given("update-passenger").uponReceiving("test-update-passenger")
.path("/passengers").method("PUT").bodyWithSingleQuotes("{'id':1,'amount':1000}", "application/json").willRespondWith().status(200)
.bodyWithSingleQuotes("{'id':1,'name':'Adam Smith','balance':5000}", "application/json").toPact();
}
@Test
@PactVerification(fragment = "callGetPassenger")
public void verifyGetPassengerPact() {
Passenger passenger = passengerManagementClient.getPassenger("test");
Assert.assertNotNull(passenger);
Assert.assertNotNull(passenger.getId());
}
@Test
@PactVerification(fragment = "callUpdatePassenger")
public void verifyUpdatePassengerPact() {
Passenger passenger = passengerManagementClient.updatePassenger(new PassengerInput(1L, 1000));
Assert.assertNotNull(passenger);
Assert.assertNotNull(passenger.getId());
}
}
Just running the tests is not enough. We also have to publish pacts generated during tests to Pact Broker. In order to achieve it we have to include the following Maven plugin to our pom.xml and then execute command mvn clean install pact:publish.
<plugin>
<groupId>au.com.dius</groupId>
<artifactId>pact-jvm-provider-maven_2.12</artifactId>
<version>3.5.21</version>
<configuration>
<pactBrokerUrl>http://192.168.99.100:9080</pactBrokerUrl>
</configuration>
</plugin>
Pact provides support for Spring on the provider side. Thanks to that we may use MockMvc controllers or inject properties from application.yml into the test class. Here’s dependency declaration that has to be included to our pom.xml
<dependency>
<groupId>au.com.dius</groupId>
<artifactId>pact-jvm-provider-spring_2.12</artifactId>
<version>3.5.21</version>
<scope>test</scope>
</dependency>
Now , the contract is being verified on the provider side. We need to pass provider name inside @Provider annotation and name of states for every verification test inside @State. These values has been during the tests on the consumer side inside @Pact annotation (fields state and provider).
@RunWith(SpringRestPactRunner.class)
@Provider("passengerManagementProvider")
@PactBroker
public class PassengerControllerContractTests {
@InjectMocks
private PassengerController controller = new PassengerController();
@Mock
private PassengerRepository repository;
@TestTarget
public final MockMvcTarget target = new MockMvcTarget();
@Before
public void before() {
MockitoAnnotations.initMocks(this);
target.setControllers(controller);
}
@State("get-passenger")
public void testGetPassenger() {
target.setRunTimes(3);
Mockito.when(repository.findByLogin(Mockito.anyString()))
.thenReturn(new Passenger(1L, "Adam Smith", "test", 4000))
.thenReturn(new Passenger(3L, "Tom Hamilton", "hamilton", 400000))
.thenReturn(new Passenger(5L, "John Scott", "scott", 222));
}
@State("update-passenger")
public void testUpdatePassenger() {
target.setRunTimes(1);
Passenger passenger = new Passenger(1L, "Adam Smith", "test", 4000);
Mockito.when(repository.findById(1L)).thenReturn(passenger);
Mockito.when(repository.update(Mockito.any(Passenger.class)))
.thenReturn(new Passenger(1L, "Adam Smith", "test", 5000));
}
}
Pact Broker host and port are injected from application.yml file.
pactbroker:
host: "192.168.99.100"
port: "8090"
Performance tests with Gatling
An important step of testing microservices before deploying them on production is performance testing. One of interesting tools in this area is Gatling. It is highly capable load testing tool written in Scala. It means that we also have to use Scala DSL in order to build test scenarios. Let’s begin from adding required library to pom.xml file.
<dependency>
<groupId>io.gatling.highcharts</groupId>
<artifactId>gatling-charts-highcharts</artifactId>
<version>2.3.1</version>
</dependency>
Now, we may proceed to the test. In the scenario visible above we are testing two endpoints exposed by trip-management: POST /trips and PUT /trips/payment/${tripId}. In fact, this scenario verifies the whole functionality of our sample system, where we are setting up trip and then pay for it after finish.
Every test class using Gatling needs to extend Simulation class. We are defining scenario using scenario method and then setting its name. We may define multiple executions inside single scenario. After every execution of POST /trips method test save generated id returned by the service. Then it inserts that id into the URL used for calling method PUT /trips/payment/${tripId}. Every single test expects response with 200 OK status.
Gatling provides two interesting features, which are worth mentioning. You can see how they are used in the following performance test. First of them is feeder. It is used for polling records and injecting their content into the test. Feed rPassengers selects one of five defined logins randomly. The final test result may be verified using Assertions API. It is responsible for verifying global statistics like response time or number of failed requests matches expectations for a whole simulation. In the scenario visible below the criterium is max response time that needs to be lower 100 milliseconds.
class CreateAndPayTripPerformanceTest extends Simulation {
val rPassengers = Iterator.continually(Map("passenger" -> List("walker","smith","hamilton","scott","holmes").lift(Random.nextInt(5)).get))
val scn = scenario("CreateAndPayTrip").feed(rPassengers).repeat(100, "n") {
exec(http("CreateTrip-API")
.post("http://localhost:8090/trips")
.header("Content-Type", "application/json")
.body(StringBody("""{"destination":"test${n}","locationX":${n},"locationY":${n},"username":"${passenger}"}"""))
.check(status.is(200), jsonPath("$.id").saveAs("tripId"))
).exec(http("PayTrip-API")
.put("http://localhost:8090/trips/payment/${tripId}")
.header("Content-Type", "application/json")
.check(status.is(200))
)
}
setUp(scn.inject(atOnceUsers(20))).maxDuration(FiniteDuration.apply(5, TimeUnit.MINUTES))
.assertions(global.responseTime.max.lt(100))
}
In order to run Gatling performance test you need to include the following Maven plugin to your pom.xml. You may run a single scenario or run multiple scenarios. After including the plugin you only need to execute command mvn clean gatling:test.
<plugin>
<groupId>io.gatling</groupId>
<artifactId>gatling-maven-plugin</artifactId>
<version>2.2.4</version>
<configuration>
<simulationClass>pl.piomin.performance.tests.CreateAndPayTripPerformanceTest</simulationClass>
</configuration>
</plugin>
Here are some diagrams illustrating result of performance tests for our microservice. Because maximum response time has been greater than set inside assertion (100ms), the test has failed.
and …
Summary
The right selection of tools is not the most important element phase of microservices testing. However, right tools can help you facing the key challenges related to it. Hoverfly allows to create full component tests that verifies if your microservice is able to handle delays or error from downstream services. Pact helps you to organize team by sharing and verifying contracts between independently developed microservices. Finally, Gatling can help you implementing load tests for selected scenarios, in order to verify an end-to-end performance of your system.
The source code used as a demo for this article is available on GitHub: https://github.com/piomin/sample-testing-microservices.git. If you find this article interesting for you you may be also interested in some other articles related to this subject:
Piotr's TechBlog 