Back to Microservice Course

Microservice Course

Circuit Breaker

Important: Netflix Hystrix is in Maintenance Mode

Hystrix entered maintenance mode in 2018 and is no longer actively developed. The final stable release was 1.5.18. For all new projects and upgrades, use Resilience4j — it is the officially recommended replacement in the Spring Cloud ecosystem and is actively maintained. The circuit breaker concepts below are fully applicable to Resilience4j; only the API and annotations differ.

Problem Statement

In a distributed system, calls to downstream services can fail due to transient faults — slow network connections, timeouts, resource exhaustion, or temporary unavailability. Most transient faults are self-healing, but some can persist for extended periods (minutes to hours). More critically, if a downstream service is slow or unresponsive, threads in the calling service pile up waiting for responses. Under sustained load, this thread exhaustion cascades upstream — a single failing downstream service can bring down the entire chain of dependent services.

Solution: The Circuit Breaker Pattern

The Circuit Breaker pattern prevents an application from repeatedly trying to call an operation that is likely to fail. Instead of letting threads hang and cascade failures propagate, the circuit breaker monitors failure rates and, when failures exceed a threshold, "opens the circuit" — immediately returning a fallback response to callers rather than attempting the failing call. After a configured wait period, it allows a limited number of probe requests through to check if the downstream service has recovered.

The Circuit Breaker is a finite state machine with three states:

  • CLOSED: Normal operation. Requests flow through. Failure rate is being monitored.
  • OPEN: Failure rate exceeded the threshold. All requests are immediately short-circuited to the fallback — no downstream calls are made. A timer runs to determine when to try again.
  • HALF-OPEN: The wait period elapsed. A limited number of probe requests are let through. If they succeed, the circuit closes. If they fail, the circuit opens again.

screenshot of the app

Circuit Breaker Process

  1. Each failed call increments the failure counter.
  2. When failures exceed the configured threshold within the rolling window, open the circuit.
  3. While open: immediately return the fallback response to all callers (fast-fail).
  4. After the sleep/wait window elapses, move to half-open.
  5. In half-open: if the next permitted call fails, re-open the circuit. If it succeeds, close it.

What a Circuit Breaker Provides

  • Fail fast: Callers get an immediate response (fallback) rather than waiting for timeouts.
  • Fallback: Returns a graceful degraded response — a cached result, a default value, or a user-friendly error message — instead of an exception.
  • Automatic recovery: Probes the downstream service periodically and closes the circuit when it recovers, without manual intervention.

screenshot of the app

Resilience4j — Current Recommended Approach

Resilience4j is a lightweight fault-tolerance library designed for Java 8+ and functional programming. Unlike Hystrix (which ran each command in its own thread pool by default), Resilience4j is built around decorators — you wrap a function/lambda with the resilience behaviour you need, and it composes cleanly.

Resilience4j provides:

  • CircuitBreaker — the pattern described above
  • Retry — automatic retry with configurable backoff (fixed, exponential, random jitter)
  • RateLimiter — limits the number of calls per time period
  • Bulkhead — limits concurrent calls to prevent a single downstream from consuming all threads
  • TimeLimiter — enforces timeouts on async calls

Maven Dependencies

<dependency>
    <groupId>io.github.resilience4j</groupId>
    <artifactId>resilience4j-spring-boot3</artifactId>
</dependency>
<!-- For Spring Boot 2.x use resilience4j-spring-boot2 -->
<dependency>
    <groupId>org.springframework.boot</groupId>
    <artifactId>spring-boot-starter-actuator</artifactId>
</dependency>
<dependency>
    <groupId>org.springframework.boot</groupId>
    <artifactId>spring-boot-starter-aop</artifactId>
</dependency>

Configuration in application.yml

resilience4j:
  circuitbreaker:
    instances:
      billingService:
        sliding-window-type: COUNT_BASED
        sliding-window-size: 5                    # evaluate last 5 calls
        failure-rate-threshold: 50                # open if ≥50% of calls fail
        wait-duration-in-open-state: 5s           # stay open for 5 seconds before half-open
        permitted-number-of-calls-in-half-open-state: 2
        automatic-transition-from-open-to-half-open-enabled: true
  retry:
    instances:
      billingService:
        max-attempts: 3
        wait-duration: 500ms
        exponential-backoff-multiplier: 2

Usage in Service Code

@Service
@Slf4j
public class OrderService {

    private final RestTemplate restTemplate;

    public OrderService(RestTemplate restTemplate) {
        this.restTemplate = restTemplate;
    }

    @CircuitBreaker(name = "billingService", fallbackMethod = "fallbackCreateOrder")
    @Retry(name = "billingService")
    public String createOrder(String orderId) {
        return restTemplate.postForObject(
            "http://billing-service/api/v1/billing/" + orderId,
            null, String.class
        );
    }

    private String fallbackCreateOrder(String orderId, Exception ex) {
        log.warn("Billing service unavailable for order {}. Reason: {}", orderId, ex.getMessage());
        return "Billing service is temporarily unavailable. Your order " + orderId +
               " has been queued and will be processed shortly.";
    }
}

Note: The fallback method must have the same return type and the same parameters as the primary method, plus an additional Exception (or a specific exception type) parameter as the last argument.

Legacy: Netflix Hystrix (Maintenance Mode — for reference only)

The Hystrix code below represents the original implementation. It is kept for reference since many existing codebases still use it, but should not be used in new projects.

Maven Dependencies (legacy)

<dependency>
    <groupId>org.springframework.cloud</groupId>
    <artifactId>spring-cloud-starter-netflix-hystrix</artifactId>
</dependency>
<dependency>
    <groupId>org.springframework.cloud</groupId>
    <artifactId>spring-cloud-starter-netflix-hystrix-dashboard</artifactId>
</dependency>
<dependency>
    <groupId>org.springframework.boot</groupId>
    <artifactId>spring-boot-starter-actuator</artifactId>
</dependency>

Main Class (legacy)

@EnableDiscoveryClient
@EnableHystrixDashboard
@EnableCircuitBreaker
@SpringBootApplication
public class OrderManagementServiceApplication {

    public static void main(String[] args) {
        SpringApplication.run(OrderManagementServiceApplication.class);
    }

    @LoadBalanced
    @Bean
    RestTemplate restTemplate(RestTemplateBuilder builder) {
        return builder.build();
    }
}

Service Method with Hystrix (legacy)

@HystrixCommand(fallbackMethod = "fallbackCreateOrder", commandProperties = {
    @HystrixProperty(name = "execution.isolation.thread.timeoutInMilliseconds", value = "5000"),
    @HystrixProperty(name = "circuitBreaker.requestVolumeThreshold", value = "5"),
    @HystrixProperty(name = "circuitBreaker.errorThresholdPercentage", value = "50"),
    @HystrixProperty(name = "circuitBreaker.sleepWindowInMilliseconds", value = "5000")
})
public String createOrder(String orderId) throws Exception {
    Map<String, String> map = new HashMap<>();
    map.put("id", orderId);
    HttpHeaders headers = new HttpHeaders();
    headers.setAccept(Arrays.asList(MediaType.APPLICATION_JSON));
    HttpEntity<String> entity = new HttpEntity<>(null, headers);
    return restTemplate.exchange(billingURL, HttpMethod.POST, entity, String.class, map).getBody();
}

private String fallbackCreateOrder(String orderId) {
    log.warn("Billing Service is down! Fallback route enabled for order: {}", orderId);
    return "No Response From Billing Service at this moment. Service will be back shortly - " + new Date();
}

Key Hystrix Properties

  • circuitBreaker.requestVolumeThreshold: Minimum number of requests in the rolling window before the circuit can trip. Circuit won't open on the first 4 failures if this is set to 5.
  • circuitBreaker.sleepWindowInMilliseconds: Time the circuit stays open before transitioning to half-open for a probe request.
  • circuitBreaker.errorThresholdPercentage: Error percentage at or above which the circuit opens.
  • execution.isolation.thread.timeoutInMilliseconds: Timeout after which Hystrix treats the call as failed and invokes the fallback.