Back to Spring Boot Course

Spring Boot Course

API Rate Limiting

API Rate Limiting is a crucial technique used to control the rate of incoming requests to an API in order to prevent abuse, ensure fair usage, and protect the backend system from being overwhelmed. With rate limiting, you can limit the number of requests a user can make to an API within a given time period.

How It Works:

  • Thresholds: A specific limit is set on the number of requests that can be made from a single IP address or API key within a predefined time window (e.g., 100 requests per minute).
  • Sliding Window Algorithm: The rate limiter uses a sliding window technique to track when requests occur. If requests are made too frequently, the API will respond with an error, typically a 429 Too Many Requests status, indicating the user has exceeded the rate limit.
  • Dynamic Configuration: The limits can be configured dynamically through application properties, allowing for flexibility in controlling usage.
  • User Experience: When the rate limit is exceeded, the system will include a Retry-After header, informing the client when they can try making another request.

Example Usage:

  • Protect your public API endpoints by applying the rate limiting policy to control usage across the entire API.
  • Implement custom rate limits for different user tiers (e.g., premium users might have higher limits than free-tier users).

How We Implement Rate Limiting

We use a Sliding Window Rate Limiter to handle rate limiting, which allows for more precise control over request limits, ensuring that even burst traffic does not exceed your defined thresholds. This solution guarantees that your API remains fast and reliable under varying usage patterns.

Let's dive into the coding process:

  • Define the Time Window: Set the maximum number of requests allowed within a specified time frame (e.g., 10 requests per minute).

  • Track Requests: Use a ConcurrentHashMap to store the timestamps of requests for each IP address.

  • Sliding Window Logic:

  1. Remove outdated timestamps (those outside the time window) when a request is received.
  2. Check if the number of remaining timestamps is within the allowed limit.
  3. If yes, process the request and log the timestamp.
  4. If no, reject the request with a "429 Too Many Requests" response.

Code Implementation

RateLimit.java

package com.praveen.ratelimiter.service;

import java.lang.annotation.ElementType;
import java.lang.annotation.Retention;
import java.lang.annotation.RetentionPolicy;
import java.lang.annotation.Target;

@Target(ElementType.METHOD)
@Retention(RetentionPolicy.RUNTIME)
public @interface RateLimit {
}

SlidingWindowRateLimiter.java

package com.praveen.ratelimiter.service;

import java.time.Instant;
import java.util.concurrent.ConcurrentHashMap;
import java.util.concurrent.ConcurrentLinkedDeque;
import java.util.concurrent.ConcurrentMap;
import java.util.concurrent.TimeUnit;

public class SlidingWindowRateLimiter {
    private final int maxRequests;
    private final long timeWindowMillis;
    private final ConcurrentMap<String, ConcurrentLinkedDeque<Instant>> requestLogs = new ConcurrentHashMap<>();

    public SlidingWindowRateLimiter(int maxRequests, long timeWindow, TimeUnit timeUnit) {
        this.maxRequests = maxRequests;
        this.timeWindowMillis = timeUnit.toMillis(timeWindow);
    }

    public boolean isAllowed(String ip) {
        Instant now = Instant.now();
        requestLogs.putIfAbsent(ip, new ConcurrentLinkedDeque<>());
        ConcurrentLinkedDeque<Instant> timestamps = requestLogs.get(ip);

        // Remote outdated requests
        synchronized (timestamps) {
            while (!timestamps.isEmpty()) {
                assert timestamps.peekFirst() != null;
                if (!now.minusMillis(timeWindowMillis).isAfter(timestamps.peekFirst())) {
                    break;
                }

                timestamps.pollFirst();
            }

            // Check if the number of requests is within the allowed limit
            if (timestamps.size() < maxRequests) {
                timestamps.addLast(now);
                return true;
            } else {
                return false;
            }
        }
    }
}

RateLimiterService.java

package com.praveen.ratelimiter.service;

import org.springframework.beans.factory.annotation.Value;
import org.springframework.stereotype.Service;

import java.util.concurrent.TimeUnit;

@Service
public class RateLimiterService {
    @Value("${rateLimiter.maxRequests}")
    private int maxRequests;

    @Value("${rateLimiter.timeWindow}")
    private int timeWindow;

    private final SlidingWindowRateLimiter rateLimiter;

    public RateLimiterService() {
        this.rateLimiter = new SlidingWindowRateLimiter(maxRequests, timeWindow, TimeUnit.MINUTES);
    }

    public boolean isAllowed(String ip) {
        return rateLimiter.isAllowed(ip);
    }
}

RateLimitingAspect.java

package com.praveen.ratelimiter.service;

import com.praveen.ratelimiter.exception.RateLimitExceededException;
import jakarta.servlet.http.HttpServletRequest;
import org.aspectj.lang.annotation.Aspect;
import org.aspectj.lang.annotation.Before;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import org.springframework.stereotype.Component;


@Aspect
@Component
public class RateLimitingAspect {

    private static final Logger logger = LoggerFactory.getLogger(RateLimitingAspect.class);
    private final HttpServletRequest httpServletRequest;
    private final RateLimiterService rateLimiterService;

    public RateLimitingAspect(HttpServletRequest httpServletRequest, RateLimiterService rateLimiterService) {
        this.httpServletRequest = httpServletRequest;
        this.rateLimiterService = rateLimiterService;
    }


    @Before("@annotation(RateLimit) && execution(public * *(..))")
    public void rateLimit() {
        String ip = httpServletRequest.getRemoteAddr();
        if (!rateLimiterService.isAllowed(ip)) {
            logger.warn("Rate limit exceeded for IP: {}", ip);
            throw new RateLimitExceededException("Too many requests from IP: " + ip);
        }
    }

}

GlobalExceptionHandler.java

package com.praveen.ratelimiter.exception;

import jakarta.servlet.http.HttpServletRequest;
import org.springframework.http.HttpStatus;
import org.springframework.http.ResponseEntity;
import org.springframework.web.bind.annotation.ControllerAdvice;
import org.springframework.web.bind.annotation.ExceptionHandler;
import org.springframework.web.bind.annotation.ResponseStatus;

@ControllerAdvice
public class GlobalExceptionHandler {

    @ExceptionHandler(RateLimitExceededException.class)
    @ResponseStatus(HttpStatus.TOO_MANY_REQUESTS)
    public ResponseEntity<String> handleRateLimitExceededException(RateLimitExceededException ex, HttpServletRequest request) {
        String retryAfter = "60";
        return ResponseEntity.status(HttpStatus.TOO_MANY_REQUESTS)
                .header("Retry-After", retryAfter)
                .body("Too many requests. Please try again later.");
    }
}

RateLimitExceededException.java

package com.praveen.ratelimiter.exception;

public class RateLimitExceededException extends RuntimeException {
    public RateLimitExceededException(String message) {
        super(message);
    }
}

TestController.java

package com.praveen.ratelimiter.controller;

import com.praveen.ratelimiter.service.RateLimit;
import org.springframework.web.bind.annotation.GetMapping;
import org.springframework.web.bind.annotation.RestController;

@RestController
public class TestController {

    @RateLimit
    @GetMapping("/api/v1/test")
    public String testEndpoint() {
        return "Request successful!";
    }
}```

`application.properties`

```jsx
spring.application.name=praveen-ratelimiter-service
server.port = 8080
rateLimiter.maxRequests=10
rateLimiter.timeWindow=1

RateLimitingApplication.java

package com.praveen.ratelimiter;

import org.springframework.boot.SpringApplication;
import org.springframework.boot.autoconfigure.SpringBootApplication;

@SpringBootApplication
public class RateLimitingApplication {

    public static void main(String[] args) {
        SpringApplication.run(RateLimitingApplication.class, args);
    }

}

Plese note this will not work out for distributed rate limiting in that case we should use redis.

Lets modify the above code to use Redis-based Rate Limiting with Redisson.

We need to add below dependencies in pom.xml

<dependency>
    <groupId>org.springframework.boot</groupId>
    <artifactId>spring-boot-starter-cache</artifactId>
</dependency>

<dependency>
    <groupId>org.redisson</groupId>
    <artifactId>redisson-spring-boot-starter</artifactId>
    <version>3.41.0</version>
</dependency>

<dependency>
    <groupId>org.springframework.boot</groupId>
    <artifactId>spring-boot-starter-data-redis</artifactId>
</dependency>

application.properties

# Redis configuration
spring.redis.host=localhost
spring.redis.port=6379
spring.redis.password= # If required
spring.redis.database=0

# Redisson specific configuration (optional but useful)
redisson.lock-watchdog-timeout=10000
redisson.threads=16
redissonnetty-pool-size=64

RateLimitingApplication.java

@SpringBootApplication
@EnableCaching
public class RateLimitingApplication {
    public static void main(String[] args) {
        SpringApplication.run(RateLimitingApplication.class, args);
    }
}

RedissonConfig.java

import org.redisson.api.RedissonClient;
import org.redisson.api.RMap;
import org.redisson.api.RList;
import org.redisson.config.Config;
import org.redisson.config.SingleServerConfig;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;

@Configuration
public class RedissonConfig {

    @Bean
    public RedissonClient redissonClient() {
        Config config = new Config();
        SingleServerConfig serverConfig = config.useSingleServer()
                                               .setAddress("redis://localhost:6379");
        return org.redisson.Redisson.create(config);
    }
}

SlidingWindowRateLimiter.java

import org.redisson.api.RedissonClient;
import org.redisson.api.RList;
import org.redisson.api.RMap;
import org.springframework.beans.factory.annotation.Value;
import org.springframework.cache.annotation.Cacheable;
import org.springframework.stereotype.Service;

import java.time.Instant;
import java.util.concurrent.TimeUnit;

@Service
public class SlidingWindowRateLimiter {

    private final RedissonClient redissonClient;
    private final int maxRequests;
    private final long timeWindowMillis;

    public SlidingWindowRateLimiter(RedissonClient redissonClient,
                                    @Value("${rateLimiter.maxRequests}") int maxRequests,
                                    @Value("${rateLimiter.timeWindow}") int timeWindow,
                                    TimeUnit timeUnit) {
        this.redissonClient = redissonClient;
        this.maxRequests = maxRequests;
        this.timeWindowMillis = timeUnit.toMillis(timeWindow);
    }

    public boolean isAllowed(String ip) {
        // Get the distributed list for storing request timestamps
        RList<Instant> timestamps = redissonClient.getList("rateLimit:" + ip);

        Instant now = Instant.now();

        // Remove outdated timestamps
        timestamps.removeIf(timestamp -> timestamp.isBefore(now.minusMillis(timeWindowMillis)));

        // If we have space for another request, allow it
        if (timestamps.size() < maxRequests) {
            timestamps.add(now); // Record the timestamp of the new request
            return true;
        } else {
            return false; // Too many requests, rate limit exceeded
        }
    }
}

RateLimitingAspect.java

import jakarta.servlet.http.HttpServletRequest;
import org.aspectj.lang.annotation.Aspect;
import org.aspectj.lang.annotation.Before;
import org.springframework.stereotype.Component;

@Aspect
@Component
public class RateLimitingAspect {

    private final HttpServletRequest httpServletRequest;
    private final SlidingWindowRateLimiter rateLimiterService;

    public RateLimitingAspect(HttpServletRequest httpServletRequest, SlidingWindowRateLimiter rateLimiterService) {
        this.httpServletRequest = httpServletRequest;
        this.rateLimiterService = rateLimiterService;
    }

    @Before("@annotation(RateLimit) && execution(public * *(..))")
    public void rateLimit() {
        String ip = httpServletRequest.getRemoteAddr();
        if (!rateLimiterService.isAllowed(ip)) {
            throw new RateLimitExceededException("Too many requests from IP: " + ip);
        }
    }
}

GlobalExceptionHandler.java

import jakarta.servlet.http.HttpServletRequest;
import org.springframework.http.HttpStatus;
import org.springframework.http.ResponseEntity;
import org.springframework.web.bind.annotation.ControllerAdvice;
import org.springframework.web.bind.annotation.ExceptionHandler;
import org.springframework.web.bind.annotation.ResponseStatus;

@ControllerAdvice
public class GlobalExceptionHandler {

    @ExceptionHandler(RateLimitExceededException.class)
    @ResponseStatus(HttpStatus.TOO_MANY_REQUESTS)
    public ResponseEntity<String> handleRateLimitExceededException(RateLimitExceededException ex, HttpServletRequest request) {
        String retryAfter = "60"; // Retry interval in seconds
        return ResponseEntity.status(HttpStatus.TOO_MANY_REQUESTS)
                .header("Retry-After", retryAfter)
                .body("Too many requests. Please try again later.");
    }
}

TestController.java

import org.springframework.web.bind.annotation.GetMapping;
import org.springframework.web.bind.annotation.RestController;

@RestController
public class TestController {

    @RateLimit
    @GetMapping("/api/v1/test")
    public String testEndpoint() {
        return "Request successful!";
    }
}

RateLimit.java

import java.lang.annotation.ElementType;
import java.lang.annotation.Retention;
import java.lang.annotation.RetentionPolicy;
import java.lang.annotation.Target;

@Target(ElementType.METHOD)
@Retention(RetentionPolicy.RUNTIME)
public @interface RateLimit {
}

This approach ensures that your API rate limiting can work consistently and efficiently in a distributed, horizontally scalable environment.

We can also use Token Bucket algorithm for rate limiting.

Token Bucket Algorithm Overview

  • Bucket: We have a bucket that holds tokens. Tokens are added to the bucket at a fixed rate (usually per second or minute).
  • Capacity: The bucket has a maximum capacity (maximum number of tokens it can hold).
  • Request Handling: When a request arrives, a token is removed from the bucket. If there are no tokens in the bucket, the request is rejected.
  • Refill Rate: The bucket refills with tokens at a steady rate until it reaches the maximum capacity.

The advantage of the token bucket algorithm is that it allows burst traffic. If the bucket has extra tokens (from previous periods when requests were lower), users can send a burst of requests without being immediately throttled.

Key Parameters

  • Token Refill Rate: The rate at which tokens are added to the bucket.
  • Bucket Capacity: The maximum number of tokens the bucket can hold.
  • Max Requests Per Time Interval: The maximum number of requests a client can make before they start getting blocked.

Steps to Implement Token Bucket

  • Redis for Distributed Rate Limiting: Redis will be used to track the token bucket state across multiple instances of the application.
  • Bucket Key: Each user/IP address will have a separate key in Redis for their token bucket.
  • Rate Limit Check: On every request, check if there are tokens available. If yes, allow the request and consume a token. If no, reject the request with a 429 status.

Code Implementation

pom.xml

<dependency>
    <groupId>org.springframework.boot</groupId>
    <artifactId>spring-boot-starter-cache</artifactId>
</dependency>

<dependency>
    <groupId>org.redisson</groupId>
    <artifactId>redisson-spring-boot-starter</artifactId>
    <version>3.41.0</version>
</dependency>

<dependency>
    <groupId>org.springframework.boot</groupId>
    <artifactId>spring-boot-starter-data-redis</artifactId>
</dependency>

RedissonConfig.java

import org.redisson.api.RedissonClient;
import org.redisson.config.Config;
import org.redisson.config.SingleServerConfig;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;

@Configuration
public class RedissonConfig {

    @Bean
    public RedissonClient redissonClient() {
        Config config = new Config();
        SingleServerConfig serverConfig = config.useSingleServer()
                                               .setAddress("redis://localhost:6379"); // Change to your Redis server address
        return org.redisson.Redisson.create(config);
    }
}

TokenBucketRateLimiter.java

import org.redisson.api.RedissonClient;
import org.redisson.api.RBucket;
import org.springframework.beans.factory.annotation.Value;
import org.springframework.stereotype.Service;

import java.time.Instant;
import java.util.concurrent.TimeUnit;

@Service
public class TokenBucketRateLimiter {

    private final RedissonClient redissonClient;
    private final int maxTokens;
    private final int refillRate;  // tokens per second
    private final long refillIntervalMillis;  // refill interval in milliseconds

    public TokenBucketRateLimiter(RedissonClient redissonClient,
                                  @Value("${rateLimiter.maxTokens}") int maxTokens,
                                  @Value("${rateLimiter.refillRate}") int refillRate) {
        this.redissonClient = redissonClient;
        this.maxTokens = maxTokens;
        this.refillRate = refillRate;
        this.refillIntervalMillis = TimeUnit.SECONDS.toMillis(1) / refillRate;  // Refill interval in ms
    }

    public boolean isAllowed(String userId) {
        String key = "rateLimit:" + userId;
        RBucket<Long> tokensBucket = redissonClient.getBucket(key + ":tokens");

        // Get current token count and timestamp of the last refill
        Long tokens = tokensBucket.get();
        if (tokens == null) {
            tokens = 0L;  // No tokens initially
        }

        // Refill tokens based on time passed
        refillTokens(userId, tokens);

        // Check if the bucket has enough tokens for the request
        if (tokens > 0) {
            // Consume a token
            tokensBucket.set(tokens - 1);
            return true;
        } else {
            // No tokens left, request is not allowed
            return false;
        }
    }

    private void refillTokens(String userId, Long currentTokens) {
        String key = "rateLimit:" + userId;
        RBucket<Long> lastRefillTimeBucket = redissonClient.getBucket(key + ":lastRefillTime");

        Long lastRefillTime = lastRefillTimeBucket.get();
        long currentTime = Instant.now().toEpochMilli();

        if (lastRefillTime == null || currentTime - lastRefillTime > refillIntervalMillis) {
            // Calculate how many tokens should have been refilled
            long refillCount = (currentTime - (lastRefillTime == null ? currentTime : lastRefillTime)) / refillIntervalMillis;

            long newTokenCount = Math.min(maxTokens, currentTokens + refillCount);
            redissonClient.getBucket(key + ":tokens").set(newTokenCount);  // Set the new token count
            lastRefillTimeBucket.set(currentTime);  // Update the last refill time
        }
    }
}

RateLimitingAspect.java

import jakarta.servlet.http.HttpServletRequest;
import org.aspectj.lang.annotation.Aspect;
import org.aspectj.lang.annotation.Before;
import org.springframework.stereotype.Component;

@Aspect
@Component
public class RateLimitingAspect {

    private final HttpServletRequest httpServletRequest;
    private final TokenBucketRateLimiter rateLimiterService;

    public RateLimitingAspect(HttpServletRequest httpServletRequest, TokenBucketRateLimiter rateLimiterService) {
        this.httpServletRequest = httpServletRequest;
        this.rateLimiterService = rateLimiterService;
    }

    @Before("@annotation(RateLimit) && execution(public * *(..))")
    public void rateLimit() {
        String ip = httpServletRequest.getRemoteAddr();
        if (!rateLimiterService.isAllowed(ip)) {
            throw new RateLimitExceededException("Too many requests from IP: " + ip);
        }
    }
}

GlobalExceptionHandler.java

import jakarta.servlet.http.HttpServletRequest;
import org.springframework.http.HttpStatus;
import org.springframework.http.ResponseEntity;
import org.springframework.web.bind.annotation.ControllerAdvice;
import org.springframework.web.bind.annotation.ExceptionHandler;

@ControllerAdvice
public class GlobalExceptionHandler {

    @ExceptionHandler(RateLimitExceededException.class)
    public ResponseEntity<String> handleRateLimitExceededException(RateLimitExceededException ex, HttpServletRequest request) {
        return ResponseEntity.status(HttpStatus.TOO_MANY_REQUESTS)
                .header("Retry-After", "60")  // Example: Retry after 60 seconds
                .body("Too many requests. Please try again later.");
    }
}

TestController.java

import org.springframework.web.bind.annotation.GetMapping;
import org.springframework.web.bind.annotation.RestController;

@RestController
public class TestController {

    @RateLimit
    @GetMapping("/api/v1/test")
    public String testEndpoint() {
        return "Request successful!";
    }
}

application.properties

rateLimiter.maxTokens=100  # Max tokens in the bucket
rateLimiter.refillRate=1  # Refill 1 token per second

The Token Bucket algorithm is ideal for managing bursts of traffic, allowing users to make requests

The decision between Token Bucket and Sliding Window rate limiting depends on the specific requirements of your application and the type of traffic patterns you're trying to control.

Token Bucket is Better When:

  • You Need to Handle Burst Traffic: Token Bucket is the go-to choice if you want to allow some flexibility for burst traffic. For example, if your API can handle short bursts of requests but needs to limit the overall rate over time, Token Bucket is ideal.
  • You Need Fairer Traffic Distribution: Token Bucket is better suited when you have a more flexible throughput pattern. Users can send requests in bursts as long as they don’t exceed the bucket size over a longer period.
  • Your System Handles Spikes Gracefully: If your system has the capacity to handle short bursts without crashing or going into overload, Token Bucket will work well because it allows tokens to accumulate for later use.

Sliding Window is Better When:

  • You Want to Prevent Bursts Entirely: If you want more predictable behavior and to avoid any sudden spikes that could overwhelm your system, Sliding Window is the better option. It enforces a steady rate of requests by limiting requests within specific time intervals.
  • You Need Strict Rate Control: Sliding Window allows for more strict control over request rates, making it ideal when your application needs to enforce a hard cap on the number of requests in a given time period, without any sudden surges.
  • You Need More Precision: If precise control over throughput is a priority (i.e., each user should only make requests at a consistent rate), Sliding Window will give you better results.

Example Use Cases:

  • Token Bucket Use Case:
  1. API for streaming or media platforms where users may experience bursts of activity (e.g., video streaming requests).
  2. Short-lived or burst-heavy applications like search engines or ad tech platforms where requests are sent in bursts but need to be controlled to prevent overload.
  • Sliding Window Use Case:
  1. API for payment gateways where strict, predictable throughput is needed to prevent excessive or fraudulent transactions.
  2. Systems that need to enforce precise rate limiting based on real-time request counts, such as login throttling, ticket booking systems, etc.

Conclusion

  • Token Bucket is ideal when you want flexibility and are willing to allow bursts within an overall rate limit.
  • Sliding Window is better when you need consistent, predictable request handling with no burstiness and stricter control over the rate.

If your system handles unpredictable bursts well and you need flexibility, go with Token Bucket. If you need more predictable, uniform control over request rates, Sliding Window would be the better choice.