Currently Spring Cloud Gateway is second the most popular Spring Cloud project just after Spring Cloud Netflix (in terms of number of stars on GitHub). It has been created as a successor of Zuul proxy in Spring Cloud family. This project provides an API Gateway for microservices architecture, and is built on top of reactive Netty and Project Reactor. It is designed to provide a simple, but effective way to route to APIs and address such popular concerns as security, monitoring/metrics, and resiliency.
Spring Cloud Gateway offers you many features and configuration options. Today I’m going to focus on the single one, but very interesting aspect of gateway configuration – rate limiting. A rate limiter may be defined as a way to control the rate of traffic sent or received on the network. We can also define a few types of rate limiting. Spring Cloud Gateway currently provides Request Rate Limiter, which is responsible for restrict each user to N requests per second.
When using RequestRateLimiter
with Spring Cloud Gateway we may leverage Redis. Spring Cloud implementation uses token bucket algorithm to do rate limiting. This algorithm has a centralized bucket host where you take tokens on each request, and slowly drip more tokens into the bucket. If the bucket is empty, it rejects the request.
1. Dependencies
We will test our sample application against rate limiting under higher traffic. First, we need to include some dependencies. Of course Spring Cloud Gateway starter is required. For handling rate limiter with Redis we also need to add dependency to spring-boot-starter-data-redis-reactive
starter. Other dependencies are used for the test purpose. Module mockserver
provided within Testcontainers. It is responsible for mocking a target service. In turn, the library mockserver-client-java
is used for integration with mockserver
container during the test. And the last library junit-benchmarks
is used for benchmarking test method and running the test concurrently.
<dependency> <groupId>org.springframework.cloud</groupId> <artifactId>spring-cloud-starter-gateway</artifactId> </dependency> <dependency> <groupId>org.springframework.boot</groupId> <artifactId>spring-boot-starter-data-redis-reactive</artifactId> </dependency> <dependency> <groupId>org.projectlombok</groupId> <artifactId>lombok</artifactId> </dependency> <dependency> <groupId>org.springframework.boot</groupId> <artifactId>spring-boot-starter-test</artifactId> <scope>test</scope> </dependency> <dependency> <groupId>org.testcontainers</groupId> <artifactId>mockserver</artifactId> <version>1.12.3</version> <scope>test</scope> </dependency> <dependency> <groupId>org.mock-server</groupId> <artifactId>mockserver-client-java</artifactId> <version>3.10.8</version> <scope>test</scope> </dependency> <dependency> <groupId>com.carrotsearch</groupId> <artifactId>junit-benchmarks</artifactId> <version>0.7.2</version> <scope>test</scope> </dependency>
The sample application is built on top of Spring Boot 2.2.1.RELEASE
and uses Spring Cloud Hoxton.RC2
Release Train.
<parent> <groupId>org.springframework.boot</groupId> <artifactId>spring-boot-starter-parent</artifactId> <version>2.2.1.RELEASE</version> </parent> <properties> <project.build.sourceEncoding>UTF-8</project.build.sourceEncoding> <project.reporting.outputEncoding>UTF-8</project.reporting.outputEncoding> <java.version>11</java.version> <spring-cloud.version>Hoxton.RC2</spring-cloud.version> </properties> <dependencyManagement> <dependencies> <dependency> <groupId>org.springframework.cloud</groupId> <artifactId>spring-cloud-dependencies</artifactId> <version>${spring-cloud.version}</version> <type>pom</type> <scope>import</scope> </dependency> </dependencies> </dependencyManagement>
2. Implementation
Request rate limiting is realized using Spring Cloud Gateway component called GatewayFilter
. Each instance of this filter is constructed in with a specific factory. Filter is of course responsible for modifying requests and responses before or after sending the downstream request. Currently, there are 30 available built-in gateway filter factories.
The GatewayFilter
takes an optional keyResolver
parameter and parameters specific to the rate limiter implementation (in that case an implementation using Redis). Parameter keyResolver
is a bean that implements the KeyResolver
interface. It allows you to apply different strategies to derive the key for limiting requests. Following Spring Cloud Gateway documentation:
The default implementation of
KeyResolver
is thePrincipalNameKeyResolver
which retrieves thePrincipal
from theServerWebExchange
and callsPrincipal.getName()
. By default, if the KeyResolver does not find a key, requests will be denied. This behavior can be adjusted with thespring.cloud.gateway.filter.request-rate-limiter.deny-empty-key
(true or false) andspring.cloud.gateway.filter.request-rate-limiter.empty-key-status-code
properties.
Since, we have discussed some theoretical aspects of rate limiting we may proceed to the implementation. First, let’s define main class and very simple KeyResolver
bean, that is always equal to one.
@SpringBootApplication public class GatewayApplication { public static void main(String[] args) { SpringApplication.run(GatewayApplication.class, args); } @Bean KeyResolver userKeyResolver() { return exchange -> Mono.just("1"); } }
Assuming we have the following configuration and a target application running on port 8091
we may perform some test calls. You may set two properties for customize the process. The redis-rate-limiter.replenishRate
decide how many requests per second a user is allowed to send without any dropped requests. This is the rate that the token bucket is filled. The second property redis-rate-limiter.burstCapacity
is the maximum number of requests a user is allowed to do in a single second. This is the number of tokens the token bucket can hold. Setting this value to zero will block all requests.
server: port: ${PORT:8085} spring: application: name: gateway-service redis: host: 192.168.99.100 port: 6379 cloud: gateway: routes: - id: account-service uri: http://localhost:8091 predicates: - Path=/account/** filters: - RewritePath=/account/(?<path>.*), /$\{path} - name: RequestRateLimiter args: redis-rate-limiter.replenishRate: 10 redis-rate-limiter.burstCapacity: 20
Now, if you call the endpoint exposed by the gateway you get the following response. It includes some specific headers, which are prefixed by x-ratelimit
. Header x-ratelimit-burst-capacity
indicates to burstCapacity
value, x-ratelimit-replenish-rate
indicates to replenishRate
value, and the most important x-ratelimit-remaining
, which shows you the number of requests you may send in the next second.
If you exceed the number of allowed requests Spring Cloud Gateway return response with code HTTP 429 - Too Many Requests
, and will not process the incoming request.
3. Testing
We have the Spring Boot test that uses two Docker containers provided by Testcontainers: MockServer
and Redis
. Because the exposed port is generated dynamically we need to set gateway properties in @BeforeClass
method before running the test. Inside init method we also use MockServerClient
to define mock service on the mock server container. Our test method is running concurrently in six threads and is repeated 600 times.
@SpringBootTest(webEnvironment = SpringBootTest.WebEnvironment.DEFINED_PORT) @RunWith(SpringRunner.class) public class GatewayRateLimiterTest { private static final Logger LOGGER = LoggerFactory.getLogger(GatewayRateLimiterTest.class); @Rule public TestRule benchmarkRun = new BenchmarkRule(); @ClassRule public static MockServerContainer mockServer = new MockServerContainer(); @ClassRule public static GenericContainer redis = new GenericContainer("redis:5.0.6").withExposedPorts(6379); @Autowired TestRestTemplate template; @BeforeClass public static void init() { System.setProperty("spring.cloud.gateway.routes[0].id", "account-service"); System.setProperty("spring.cloud.gateway.routes[0].uri", "http://192.168.99.100:" + mockServer.getServerPort()); System.setProperty("spring.cloud.gateway.routes[0].predicates[0]", "Path=/account/**"); System.setProperty("spring.cloud.gateway.routes[0].filters[0]", "RewritePath=/account/(?<path>.*), /$\\{path}"); System.setProperty("spring.cloud.gateway.routes[0].filters[1].name", "RequestRateLimiter"); System.setProperty("spring.cloud.gateway.routes[0].filters[1].args.redis-rate-limiter.replenishRate", "10"); System.setProperty("spring.cloud.gateway.routes[0].filters[1].args.redis-rate-limiter.burstCapacity", "20"); System.setProperty("spring.redis.host", "192.168.99.100"); System.setProperty("spring.redis.port", "" + redis.getMappedPort(6379)); new MockServerClient(mockServer.getContainerIpAddress(), mockServer.getServerPort()) .when(HttpRequest.request() .withPath("/1")) .respond(response() .withBody("{\"id\":1,\"number\":\"1234567890\"}") .withHeader("Content-Type", "application/json")); } @Test @BenchmarkOptions(warmupRounds = 0, concurrency = 6, benchmarkRounds = 600) public void testAccountService() { ResponseEntity<Account> r = template.exchange("/account/{id}", HttpMethod.GET, null, Account.class, 1); LOGGER.info("Received: status->{}, payload->{}, remaining->{}", r.getStatusCodeValue(), r.getBody(), r.getHeaders().get("X-RateLimit-Remaining")); } }
Let’s take a look on the test result. After starting gateway allows a user to send max 20 requests in a single second. After exceeding this value it starts to return HTTP 429
.
After dropping some incoming requests gateway starts to accept them in the next second. But this time it allows to process only 10 requests, which is equal to replenishRate
parameter value.
The source code is available in GitHub repository: https://github.com/piomin/sample-spring-cloud-gateway.git.
I am testing the rate limit feature using postman. I check the response header and see that X-RateLimit-Remaining is -1. Why is it negative? Of course, rate limit is not working.
LikeLike
You should run JUnit test that performs such test. I don’t understand what you mean by testing it with postman, since I implemented this mechanism only inside unit test, not at the level of application.
LikeLike
So, we are saying I could not use your ideas in a real application? I want to use the rate limit feature in a real application. What else I need to do? I trust JUnit test will pass.
LikeLike
Hi. Take a look on my video course https://www.youtube.com/watch?v=XIkSWHX38Tg. You also have a repository with sample source code: https://github.com/piomin/course-spring-microservices.git.
LikeLike
I didn’t. I just don’t know how did you run it. What parameters have you exactly set? how many requests did you send? Share your code with me maybe
LikeLike
I tried to implement your solution. but each time I have: X-RateLimit-Remaining: -1.
and the following error in my console: org.springframework.data.redis.RedisConnectionFailureException: Unable to connect to Redis; nested exception is io.lettuce.core.RedisConnectionException: Unable to connect to 127.0.0.1:6379
LikeLike
Did you run Redis?
LikeLike
No. I just added the radish dependency to my project.
LikeLike
In my test I used testcontainers to start Redis.
LikeLike
Ok. I think it’s because I didn’t run redis.
thank you for your help.
LikeLiked by 1 person
I tried integrating it with my project, but i am getting this error
java.lang.IllegalArgumentException: Unable to find GatewayFilterFactory with name RequestRateLimiter
Kind help if I am missing any thing.
LikeLike
Which version of Spring Cloud do you use in your project?
LikeLike
First of all thanks for this great tutorial, Piotr!
I have a question, can we increase replenishRate to be per 10 seconds or per minute or more based on the route? Or generally increase this rate?
I tried to search a little bit and found that there is a redis lua script inside spring cloud gateway source code but I don’t know lua and I am not sure that would be the way to go to edit the window of replenishRate.
LikeLike