I have already written many articles, where I was using Docker containers for running some third-party solutions integrated with my sample applications. Building integration tests for such applications may not be an easy task without Docker containers. Especially, if our application integrates with databases, message brokers or some other popular tools. If you are planning to build such integration tests you should definitely take a look on Testcontainers (https://www.testcontainers.org/). Testcontainers is a Java library that supports JUnit tests, providing fast and lightweight way for running instances of common databases, Selenium web browsers, or anything else that can run in a Docker container. It provides modules for the most popular relational and NoSQL databases like Postgres, MySQL, Cassandra or Neo4j. It also allows to run popular products like Elasticsearch, Kafka, Nginx or HashiCorp’s Vault. Today I’m going to show you more advanced sample of JUnit tests that use Testcontainers to check out an integration between Spring Boot/Spring Cloud application, Postgres database and Vault. For the purposes of that example we will use the case described in one of my previous articles Secure Spring Cloud Microservices with Vault and Nomad. Let us recall that use case.
I described there how to use very interesting Vault feature called secret engines for generating database user credentials dynamically. I used Spring Cloud Vault module in my Spring Boot application to automatically integrate with that feature of Vault. The implemented mechanism is pretty easy. The application calls Vault secret engine before it tries to connect to Postgres database on startup. Vault is integrated with Postgres via secret engine, and that’s why it creates user with sufficient privileges on Postgres. Then, generated credentials are automatically injected into auto-configured Spring Boot properties used for connecting with database spring.datasource.username
and spring.datasource.password
. The following picture illustrates described solution.
Ok, we know how it works, now the question is how to automatically test it. With Testcontainers it is possible with just a few lines of code.
1. Building application
Let’s begin from a short intro to the application code. It is very simple. Here’s the list of dependencies required for building application that exposes REST API, and integrates with Postgres and Vault.
<dependency> <groupId>org.springframework.boot</groupId> <artifactId>spring-boot-starter-web</artifactId> </dependency> <dependency> <groupId>org.springframework.cloud</groupId> <artifactId>spring-cloud-starter-vault-config</artifactId> </dependency> <dependency> <groupId>org.springframework.cloud</groupId> <artifactId>spring-cloud-vault-config-databases</artifactId> </dependency> <dependency> <groupId>org.springframework.boot</groupId> <artifactId>spring-boot-starter-data-jpa</artifactId> </dependency> <dependency> <groupId>org.postgresql</groupId> <artifactId>postgresql</artifactId> <version>42.2.5</version> </dependency>
Application connects to Postgres, enables integration with Vault via Spring Cloud Vault, and automatically creates/updates tables on startup.
spring: application: name: callme-service cloud: vault: uri: http://192.168.99.100:8200 token: ${VAULT_TOKEN} postgresql: enabled: true role: default backend: database datasource: url: jdbc:postgresql://192.168.99.100:5432/postgres jpa.hibernate.ddl-auto: update
It exposes the single endpoint. The following method is responsible for handling incoming requests. It just insert a record to database and return response with app name, version and id of inserted record.
@RestController @RequestMapping("/callme") public class CallmeController { private static final Logger LOGGER = LoggerFactory.getLogger(CallmeController.class); @Autowired Optional<BuildProperties> buildProperties; @Autowired CallmeRepository repository; @GetMapping("/message/{message}") public String ping(@PathVariable("message") String message) { Callme c = repository.save(new Callme(message, new Date())); if (buildProperties.isPresent()) { BuildProperties infoProperties = buildProperties.get(); LOGGER.info("Ping: name={}, version={}", infoProperties.getName(), infoProperties.getVersion()); return infoProperties.getName() + ":" + infoProperties.getVersion() + ":" + c.getId(); } else { return "callme-service:" + c.getId(); } } }
2. Enabling Testcontainers
To enable Testcontainers for our project we need to include some dependencies to our Maven pom.xml
. We have dedicated modules for Postgres and Vault. We also include Spring Boot Test dependency, because we would like to test the whole Spring Boot app.
<dependency> <groupId>org.springframework.boot</groupId> <artifactId>spring-boot-starter-test</artifactId> <scope>test</scope> </dependency> <dependency> <groupId>org.testcontainers</groupId> <artifactId>vault</artifactId> <version>1.10.5</version> <scope>test</scope> </dependency> <dependency> <groupId>org.testcontainers</groupId> <artifactId>testcontainers</artifactId> <version>1.10.5</version> <scope>test</scope> </dependency> <dependency> <groupId>org.testcontainers</groupId> <artifactId>postgresql</artifactId> <version>1.10.5</version> <scope>test</scope> </dependency>
3. Running Vault test container
Testcontainers framework supports JUnit 4/JUnit 5 and Spock. The Vault container can be started before tests if it is annotated with @Rule
or @ClassRule
. By default it uses version 0.7
, but we can override it with newest version, which is 1.0.2
. We also may set a root token, which is then required by Spring Cloud Vault for integration with Vault.
@ClassRule public static VaultContainer vaultContainer = new VaultContainer<>("vault:1.0.2") .withVaultToken("123456") .withVaultPort(8200);
That root token can be overridden before starting JUnit test on the test class.
@RunWith(SpringRunner.class) @SpringBootTest(webEnvironment = SpringBootTest.WebEnvironment.RANDOM_PORT, properties = { "spring.cloud.vault.token=123456" }) public class CallmeTest { ... }
4. Running Postgres test container
As an alternative to @ClassRule
, we can manually start the container in a @BeforeClass
or @Before
method in the test. With this approach you will also have to stop it manually in @AfterClass
or @After
method. We start Postgres container manually, because by default it is exposed on dynamically generated port, which need to be set for Spring Boot application before starting the test. The listen port is returned by method getFirstMappedPort
invoked on PostgreSQLContainer
.
private static PostgreSQLContainer postgresContainer = new PostgreSQLContainer() .withDatabaseName("postgres") .withUsername("postgres") .withPassword("postgres123"); @BeforeClass public static void init() throws IOException, InterruptedException { postgresContainer.start(); int port = postgresContainer.getFirstMappedPort(); System.setProperty("spring.datasource.url", String.format("jdbc:postgresql://192.168.99.100:%d/postgres", postgresContainer.getFirstMappedPort())); // ... } @AfterClass public static void shutdown() { postgresContainer.stop(); }
5. Integrating Vault and Postgres containers
Once we have succesfully started both Vault and Postgres containers, we need to integrate them via Vault secret engine. First, we need to enable database secret engine Vault. After that we must configure connection to Postgres. The last step is is to configure a role. A role is a logical name that maps to a policy used to generated those credentials. All these actions may be performed using Vault commands. You can launch command on Vault container using execInContainer
method. Vault configuration commands should be executed just after Postgres container startup.
@BeforeClass public static void init() throws IOException, InterruptedException { postgresContainer.start(); int port = postgresContainer.getFirstMappedPort(); System.setProperty("spring.datasource.url", String.format("jdbc:postgresql://192.168.99.100:%d/postgres", postgresContainer.getFirstMappedPort())); vaultContainer.execInContainer("vault", "secrets", "enable", "database"); String url = String.format("connection_url=postgresql://{{username}}:{{password}}@192.168.99.100:%d?sslmode=disable", port); vaultContainer.execInContainer("vault", "write", "database/config/postgres", "plugin_name=postgresql-database-plugin", "allowed_roles=default", url, "username=postgres", "password=postgres123"); vaultContainer.execInContainer("vault", "write", "database/roles/default", "db_name=postgres", "creation_statements=CREATE ROLE \"{{name}}\" WITH LOGIN PASSWORD '{{password}}' VALID UNTIL '{{expiration}}';GRANT SELECT, UPDATE, INSERT ON ALL TABLES IN SCHEMA public TO \"{{name}}\";GRANT USAGE, SELECT ON ALL SEQUENCES IN SCHEMA public TO \"{{name}}\";", "default_ttl=1h", "max_ttl=24h"); }
6. Running application tests
Finally, we may run application tests. We just call the single endpoint exposed by the app using TestRestTemplate
, and verify the output.
@Autowired TestRestTemplate template; @Test public void test() { String res = template.getForObject("/callme/message/{message}", String.class, "Test"); Assert.assertNotNull(res); Assert.assertTrue(res.endsWith("1")); }
If you are interested what exactly happens during the test you can set a breakpoint inside test method and execute docker ps
command manually.