Reactive Logging With Spring WebFlux and Logstash

I have already introduced my Spring Boot library for synchronous HTTP request/response logging in one of my previous articles Logging with Spring Boot and Elastic Stack. This library is dedicated for synchronous REST applications built with Spring MVC and Spring Web. Since version 5.0 Spring Framework also offers support for reactive REST API through Spring WebFlux project. I decided to extend support for logging in my library to reactive Spring WebFlux.

Continue reading “Reactive Logging With Spring WebFlux and Logstash”

Advertisements

Using New Spring Cloud Load Balancer In Microservices Communication

Almost a year ago Spring Cloud has announced that most of Spring Cloud Netflix OSS projects will be moved to the maintenance mode starting from Spring Cloud Greenwich Release Train. In fact the maintenance mode only does not include Eureka, which still will be supported. I referred to that information in one of my previous articles The Future of Spring Cloud Microservices After Netflix Era. I have shared there some opinions about future of microservices with Spring Cloud. Of course, I also included an example of building microservices architecture without Netflix OSS using HashiCorp’s Consul, Spring Cloud Gateway and an early version of Spring Cloud LoadBalancer.

Continue reading “Using New Spring Cloud Load Balancer In Microservices Communication”

Part 1: Testing Kafka Microservices With Micronaut

I have already described how to build microservices architecture entirely based on message-driven communication through Apache Kafka in one of my previous articles Kafka In Microservices With Micronaut. As you can see in the article title the sample applications and integration with Kafka has been built on top of Micronaut Framework. I described some interesting features of Micronaut, that can be used for building message-driven microservices, but I specially didn’t write anything about testing. In this article I’m going to show you how to test your Kafka microservice using Micronaut Test core features (Component Tests), Testcontainers (Integration Tests) and Pact (Contract Tests).

Continue reading “Part 1: Testing Kafka Microservices With Micronaut”

Overview of Java Stream API Extensions

Stream API, which has been introduced in Java 8, is probably still the most important new feature that has been included to Java during last several years. I think that every Java developer has an opportunity to use Java Stream API in his career. Or I should rather told that you probably use it on a day-to-day basis. However, if you compare the built-in features offered for functional programming with some other languages – for example Kotlin – you will quickly realize that the number of methods provided by Stream API is very limited. Therefore, the community has created several libraries used just for extending API offered by pure Java. Today I’m going to show the most interesting Stream API extensions offered by the three popular Java libraries: StreamEx, jOOλ and Guava.

Continue reading “Overview of Java Stream API Extensions”

Using logstash-logging-spring-boot-starter for logging with Spring Boot and Logstash

I have already described some implementation details related to my library logstash-logging-spring-boot-starter for HTTP request/response logging in one of the previous articles Logging with Spring Boot and Elastic Stack. The article has been published some weeks ago, and since that time some important features has been added to this library. Today I’m going to summarise all those changes and describe all the features provided by the library.

Continue reading “Using logstash-logging-spring-boot-starter for logging with Spring Boot and Logstash”

Deploying Spring Boot Application on OpenShift with Dekorate

More advanced deployments to Kubernetes or OpenShift are a bit troublesome for developers. In comparison to Kubernetes OpenShift provides S2I (Source-2-Image) mechanism, which may help reduce a time required for preparation of application deployment descriptors. Although S2I is quite useful for developers, it solves only simple use cases and does not provide unified approach to building deployment configuration from a source code. Dekorate (https://dekorate.io), the recently created open-source project, tries to solve that problem. This project seems to be very interesting. It appears to be confirmed by RedHat, which has already announced a decision on including Dekorate to Red Hat OpenShift Application Runtimes as a “Tech Preview”. You can read more about it in this article: https://developers.redhat.com/blog/2019/08/15/how-to-use-dekorate-to-create-kubernetes-manifests.

Continue reading “Deploying Spring Boot Application on OpenShift with Dekorate”

Quick Guide to Microservices with Quarkus on Openshift

You had an opportunity to read many articles about building microservices with such frameworks like Spring Boot or Micronaut on my blog. There is another one very interesting framework dedicated for microservices architecture, which is becoming increasing popular – Quarkus. It is being introduced as a next-generation Kubernetes/Openshift native Java framework. It is built on top of well-known Java standards like CDI, JAX-RS and Eclipse MicroProfile which distinguishes it from Spring Boot.
Some other features that may convince you to use Quarkus are extremely fast boot time, minimal memory footprint optimized for running in containers, and lower time-to-first-request. Also, despite the fact that it is relatively new framework (the current version is 0.21), it has a lot of extensions including support for Hibernate, Kafka, RabbitMQ, Openapi, Vert.x and many more.
In this article I’m going to guide you through building microservices with Quarkus and running them on OpenShift (via Minishift). We will cover the following topics:

  • Building REST-based application with input validation
  • Communication between microservices with RestClient
  • Exposing health checks (liveness, readiness)
  • Exposing OpenAPI/Swagger documentation
  • Running applications on the local machine with Quarkus Maven plugin
  • Testing with JUnit and RestAssured
  • Deploying and running Quarkus applications on Minishift using source-2-image

1. Creating application – dependencies

When creating a new application you may execute a single Maven command that uses quarkus-maven-plugin. A list of dependencies should be declared in parameter -Dextensions.

mvn io.quarkus:quarkus-maven-plugin:0.21.1:create \
    -DprojectGroupId=pl.piomin.services \
    -DprojectArtifactId=employee-service \
    -DclassName="pl.piomin.services.employee.controller.EmployeeController" \
    -Dpath="/employees" \
    -Dextensions="resteasy-jackson, hibernate-validator"

Here’s the structure of our pom.xml:

<properties>
	<quarkus.version>0.21.1</quarkus.version>
	<project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
	<maven.compiler.source>11</maven.compiler.source>
	<maven.compiler.target>11</maven.compiler.target>
</properties>
<dependencyManagement>
	<dependencies>
		<dependency>
			<groupId>io.quarkus</groupId>
			<artifactId>quarkus-bom</artifactId>
			<version>${quarkus.version}</version>
			<type>pom</type>
			<scope>import</scope>
		</dependency>
	</dependencies>
</dependencyManagement>
<build>
	<plugins>
		<plugin>
			<groupId>io.quarkus</groupId>
			<artifactId>quarkus-maven-plugin</artifactId>
			<version>${quarkus.version}</version>
			<executions>
				<execution>
					<goals>
						<goal>build</goal>
					</goals>
				</execution>
			</executions>
		</plugin>
	</plugins>
</build>

For building simple REST application with input validation we don’t need many modules. As you have probably noticed I declared just two extensions, which is same as the following list of dependencies in pom.xml:

<dependency>
	<groupId>io.quarkus</groupId>
	<artifactId>quarkus-resteasy-jackson</artifactId>
</dependency>
<dependency>
	<groupId>io.quarkus</groupId>
	<artifactId>quarkus-hibernate-validator</artifactId>
</dependency>

2. Creating application – code

What might be a bit surprising for Spring Boot or Micronaut users there is no main, runnable class with static <codemain method. A resource/controller class is defacto the main class. Quarkus resource/controller class and methods should be marked using annotations from javax.ws.rs library. Here’s the implementation of REST controller inside employee-service:

@Path("/employees")
@Produces(MediaType.APPLICATION_JSON)
public class EmployeeController {

	private static final Logger LOGGER = LoggerFactory.getLogger(EmployeeController.class);
	
	@Inject
	EmployeeRepository repository;
	
	@POST
	public Employee add(@Valid Employee employee) {
		LOGGER.info("Employee add: {}", employee);
		return repository.add(employee);
	}
	
	@Path("/{id}")
	@GET
	public Employee findById(@PathParam("id") Long id) {
		LOGGER.info("Employee find: id={}", id);
		return repository.findById(id);
	}

	@GET
	public Set<Employee> findAll() {
		LOGGER.info("Employee find");
		return repository.findAll();
	}
	
	@Path("/department/{departmentId}")
	@GET
	public Set<Employee> findByDepartment(@PathParam("departmentId") Long departmentId) {
		LOGGER.info("Employee find: departmentId={}", departmentId);
		return repository.findByDepartment(departmentId);
	}
	
	@Path("/organization/{organizationId}")
	@GET
	public Set<Employee> findByOrganization(@PathParam("organizationId") Long organizationId) {
		LOGGER.info("Employee find: organizationId={}", organizationId);
		return repository.findByOrganization(organizationId);
	}
	
}

We use CDI for dependency injection and SLF4J for logging. Controller class uses in-memory repository bean for storing and retrieving data. Repository bean is annotated with CDI @ApplicationScoped and injected into the controller:

@ApplicationScoped
public class EmployeeRepository {

	private Set<Employee> employees = new HashSet<>();

	public EmployeeRepository() {
		add(new Employee(1L, 1L, "John Smith", 30, "Developer"));
		add(new Employee(1L, 1L, "Paul Walker", 40, "Architect"));
	}

	public Employee add(Employee employee) {
		employee.setId((long) (employees.size()+1));
		employees.add(employee);
		return employee;
	}
	
	public Employee findById(Long id) {
		Optional<Employee> employee = employees.stream().filter(a -> a.getId().equals(id)).findFirst();
		if (employee.isPresent())
			return employee.get();
		else
			return null;
	}

	public Set<Employee> findAll() {
		return employees;
	}
	
	public Set<Employee> findByDepartment(Long departmentId) {
		return employees.stream().filter(a -> a.getDepartmentId().equals(departmentId)).collect(Collectors.toSet());
	}
	
	public Set<Employee> findByOrganization(Long organizationId) {
		return employees.stream().filter(a -> a.getOrganizationId().equals(organizationId)).collect(Collectors.toSet());
	}

}

And the last component is domain class with validation:

public class Employee {

	private Long id;
	@NotNull
	private Long organizationId;
	@NotNull
	private Long departmentId;
	@NotBlank
	private String name;
	@Min(1)
	@Max(100)
	private int age;
	@NotBlank
	private String position;
	
	// ... GETTERS AND SETTERS
	
}

3. Unit Testing

As for the most of popular Java frameworks unit testing with Quarkus is very simple. If you are testing REST-based web application you should include the following dependencies in your pom.xml:

<dependency>
	<groupId>io.quarkus</groupId>
	<artifactId>quarkus-junit5</artifactId>
	<scope>test</scope>
</dependency>
<dependency>
	<groupId>io.rest-assured</groupId>
	<artifactId>rest-assured</artifactId>
	<scope>test</scope>
</dependency>

Let’s analyze the test class from organization-service (our another microservice along with employee-service and department-service). A test class should be annotated with @QuarkusTest. We may inject other beans via @Inject annotation. The rest is typical for JUnit and RestAssured – we are testing the API methods exposed by the controller. Because we are using in-memory repository we don’t have to mock anything except inter-service communication (we discuss it later in that article). We have some positive scenarios for GET, POST methods and a single negative scenario that does not pass input validation (testInvalidAdd).

@QuarkusTest
public class OrganizationControllerTests {

    @Inject
    OrganizationRepository repository;

    @Test
    public void testFindAll() {
        given().when().get("/organizations").then().statusCode(200).body(notNullValue());
    }

    @Test
    public void testFindById() {
        Organization organization = new Organization("Test3", "Address3");
        organization = repository.add(organization);
        given().when().get("/organizations/{id}", organization.getId()).then().statusCode(200)
                .body("id", equalTo(organization.getId().intValue()))
                .body("name", equalTo(organization.getName()));
    }

    @Test
    public void testFindByIdWithDepartments() {
        given().when().get("/organizations/{id}/with-departments", 1L).then().statusCode(200)
                .body(notNullValue())
                .body("departments.size()", is(1));
    }

    @Test
    public void testAdd() {
        Organization organization = new Organization("Test5", "Address5");
        given().contentType("application/json").body(organization)
                .when().post("/organizations").then().statusCode(200)
                .body("id", notNullValue())
                .body("name", equalTo(organization.getName()));
    }

    @Test
    public void testInvalidAdd() {
        Organization organization = new Organization();
        given().contentType("application/json").body(organization).when().post("/organizations").then().statusCode(400);
    }

}

4. Inter-service communication

Since Quarkus is targeted for running on Kubernetes it does not provide any built-in support for third-party service discovery (for example through Consul or Netflix Eureka) and HTTP client integrated with this discovery. However, Quarkus provides dedicated client support for REST communication. To use it we first need to include the following dependency:

<dependency>
	<groupId>io.quarkus</groupId>
	<artifactId>quarkus-rest-client</artifactId>
</dependency>

Quarkus provides declarative REST client based on MicroProfile REST Client. You need to create an interface with required methods and annotate it with @RegisterRestClient. Other annotations are pretty the same as on the server side. Since you use @RegisterRestClient for marking Quarkus know that this interface is meant to be available for CDI injection as a REST Client.

@Path("/departments")
@RegisterRestClient
public interface DepartmentClient {

	@GET
	@Path("/organization/{organizationId}")
	@Produces(MediaType.APPLICATION_JSON)
	List<Department> findByOrganization(@PathParam("organizationId") Long organizationId);

	@GET
	@Path("/organization/{organizationId}/with-employees")
	@Produces(MediaType.APPLICATION_JSON)
	List<Department> findByOrganizationWithEmployees(@PathParam("organizationId") Long organizationId);
	
}

Now, let’s take a look on the controller class inside organization-service. Together with @Inject we need to use @RestClient annotation to inject REST client bean properly. After that you can use interface methods to call endpoints exposed by other services.

@Path("/organizations")
@Produces(MediaType.APPLICATION_JSON)
public class OrganizationController {

	private static final Logger LOGGER = LoggerFactory.getLogger(OrganizationController.class);
	
	@Inject
	OrganizationRepository repository;
	@Inject
	@RestClient
	DepartmentClient departmentClient;
	@Inject
	@RestClient
	EmployeeClient employeeClient;
	
	// ... OTHER FIND METHODS

	@Path("/{id}/with-departments")
	@GET
	public Organization findByIdWithDepartments(@PathParam("id") Long id) {
		LOGGER.info("Organization find: id={}", id);
		Organization organization = repository.findById(id);
		organization.setDepartments(departmentClient.findByOrganization(organization.getId()));
		return organization;
	}
	
	@Path("/{id}/with-departments-and-employees")
	@GET
	public Organization findByIdWithDepartmentsAndEmployees(@PathParam("id") Long id) {
		LOGGER.info("Organization find: id={}", id);
		Organization organization = repository.findById(id);
		organization.setDepartments(departmentClient.findByOrganizationWithEmployees(organization.getId()));
		return organization;
	}
	
	@Path("/{id}/with-employees")
	@GET
	public Organization findByIdWithEmployees(@PathParam("id") Long id) {
		LOGGER.info("Organization find: id={}", id);
		Organization organization = repository.findById(id);
		organization.setEmployees(employeeClient.findByOrganization(organization.getId()));
		return organization;
	}
	
}

The last missing thing required for communication are the addresses of target services. We may provide them using field baseUri of @RegisterRestClient annotation. However, it seems that a better solution would be to place them inside application.properties. The name of property needs to contain fully qualified name of client interface and suffix mp-rest/url.

pl.piomin.services.organization.client.DepartmentClient/mp-rest/url=http://localhost:8090
pl.piomin.services.organization.client.EmployeeClient/mp-rest/url=http://localhost:8080

I have already mentioned about unit testing and inter-service communication in the previous section. To test API method that communicates with other applications we need to mock REST client. Here’s the sample of mock created for DepartmentClient. It should be visible only during the tests, so we have to place it inside src/test/java. If we annotate it with @Mock and @RestClient Quarkus automatically use this bean by default instead of declarative REST client defined inside src/main/java.

@Mock
@ApplicationScoped
@RestClient
public class MockDepartmentClient implements DepartmentClient {

    @Override
    public List<Department> findByOrganization(Long organizationId) {
        return Collections.singletonList(new Department("Test1"));
    }

    @Override
    public List<Department> findByOrganizationWithEmployees(Long organizationId) {
        return null;
    }

}

5. Monitoring and Documentation

We can easily expose health checks or API documentation with Quarkus. API documentation is built using OpenAPI/Swagger. Quarkus leverages libraries available within the project SmallRye. We should include the following dependencies to our pom.xml:

<dependency>
	<groupId>io.quarkus</groupId>
	<artifactId>quarkus-smallrye-openapi</artifactId>
</dependency>
<dependency>
	<groupId>io.quarkus</groupId>
	<artifactId>quarkus-smallrye-health</artifactId>
</dependency>

We can define two types of health checks: readiness and liveness. There are available under /health/ready and /health/live context paths. To expose them outside application we need to define a bean that implements MicroProfile HealthCheck interface. Readiness endpoint should be annotated with @Readiness, while liveness with @Liveness.

@ApplicationScoped
@Readiness
public class ReadinessHealthcheck implements HealthCheck {

    @Override
    public HealthCheckResponse call() {
        return HealthCheckResponse.named("Employee Health Check").up().build();
    }

}

To enable Swagger documentation we don’t need to do anything more than adding a dependency. Quarkus also provides built-in UI for Swagger. By default it is enabled on development mode, so if you are willing to use on the production you should add the line quarkus.swagger-ui.always-include=true to your application.properties file. Now, if run the application employee-service locally in development mode by executing Maven command mvn compile quarkus:dev you may view API specification available under URL http://localhost:8080/swagger-ui.

quarkus-swagger

Here’s my log from application startup. It prints listening port and list of loaded extensions.

quarkus-startup

6. Running Microservices on the Local Machine

Because we would like to run more than one application on the same machine we need to override their default HTTP listening port. While employee-service still running on the default 8080 port, other microservices use different ports as shown below.

department-service:
quarkus-port-department

organization-service:
quarkus-port-organization

Let’s test an inter-service communication from Swagger UI. I called endpoint GET /organizations/{id}/with-departments that calls endpoint GET /departments/organization/{organizationId} exposed by department-service. The result is visible on the below.

quarkus-communication

7. Running Microservices on OpenShift

We have already finished the implementation of our sample microservices architecture and run them on the local machine. Now, we can proceed to the last step and try to deploy these applications on Minishift. We have some different approaches when deploying Quarkus application on OpenShift. Today I’ll show you leverage the S2I build mechanism for that.
We are going to use Quarkus GraalVM Native S2I Builder. It is available on quai.io as quarkus/ubi-quarkus-native-s2i. Of course, before deploying our applications we need to start Minishift. Following Quarkus documentation GraalVM-based native build consumes much memory and CPU, so I decided to set 6GB and 4 cores for Minishift.

$ minishift start --vm-driver=virtualbox --memory=6G --cpus=4

Also, we need to modify source code of our application a little. As you probably remember we used JDK 11 for running them locally. Quarkus S2I builder supports only JDK 8, so we need to change it in our pom.xml. We also need to include a declaration of native profile as shown below:

<properties>
	<quarkus.version>0.21.1</quarkus.version>
	<project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
	<maven.compiler.source>1.8</maven.compiler.source>
	<maven.compiler.target>1.8</maven.compiler.target>
</properties>
...
<profiles>
	<profile>
		<id>native</id>
		<activation>
			<property>
				<name>native</name>
			</property>
		</activation>
		<build>
			<plugins>
				<plugin>
					<groupId>io.quarkus</groupId>
					<artifactId>quarkus-maven-plugin</artifactId>
					<version>${quarkus.version}</version>
					<executions>
						<execution>
							<goals>
								<goal>native-image</goal>
							</goals>
							<configuration>
								<enableHttpUrlHandler>true</enableHttpUrlHandler>
							</configuration>
						</execution>
					</executions>
				</plugin>
				<plugin>
					<artifactId>maven-failsafe-plugin</artifactId>
					<version>2.22.1</version>
					<executions>
						<execution>
							<goals>
								<goal>integration-test</goal>
								<goal>verify</goal>
							</goals>
							<configuration>
								<systemProperties>
									<native.image.path>${project.build.directory}/${project.build.finalName}-runner</native.image.path>
								</systemProperties>
							</configuration>
						</execution>
					</executions>
				</plugin>
			</plugins>
		</build>
	</profile>
</profiles>

Two other changes should be performed inside application.properties file. We don’t have to override port number, since Minishift dynamically assign virtual IP for every pod. An inter-service communication is realized via OpenShift discovery, so we just need to set the name of service instead of localhost.

quarkus.swagger-ui.always-include=true
pl.piomin.services.organization.client.DepartmentClient/mp-rest/url=http://department:8080
pl.piomin.services.organization.client.EmployeeClient/mp-rest/url=http://employee:8080

Finally we may deploy our applications on Minishift. To do that you should execute the following commands using your oc client:

$ oc new-app quay.io/quarkus/ubi-quarkus-native-s2i:19.1.1~https://github.com/piomin/sample-quarkus-microservices.git#openshift --context-dir=employee --name=employee
$ oc new-app quay.io/quarkus/ubi-quarkus-native-s2i:19.1.1~https://github.com/piomin/sample-quarkus-microservices.git#openshift --context-dir=department --name=department
$ oc new-app quay.io/quarkus/ubi-quarkus-native-s2i:19.1.1~https://github.com/piomin/sample-quarkus-microservices.git#openshift --context-dir=organization --name=organization

As you can see the repository with applications source code is available on my GitHub account under address https://github.com/piomin/sample-quarkus-microservices.git. The version for running on Minishift has been shared within branch openshift. The version for running on the local machine is available on master branch. Because all the applications are stored within a single repository we need to define a parameter context-dir for every single deployment.
I was quite disappointed. Although setting more memory and CPU for minishift my builds have taken a very long time – about 25 minutes.

quarkus-builds

However, finally after a long wait all my applications has been deployed.

quarkus-openshift-overview

I exposed them outside Minishift by executing the commands visible below. They can be tested using OpenShift route available under DNS http://${APP_NAME}-myproject.192.168.99.100.nip.io.

$ oc expose svc employee
$ oc expose svc department
$ oc expose svc organization

Additionally you may enable readiness and liveness health checks on OpenShift, since they are disabled by default.

quarkus-health