Circuit Breaker Pattern with Microservices Architecture

Microservices are an architectural and organizational approach to software development where software is composed of small independent services that communicate over well-defined APIs. These services are owned by small, self-contained teams. Microservices architectures make applications easier to scale and faster to develop, enabling innovation and accelerating time-to-market for new features.

In such architecture, services have to communicate with each other. Sometimes, there can be service failures, or it takes significant time to respond to requests due to network connectivity issues. The circuit breaker pattern is a design pattern that allows to prevent cascading failures in microservices architecture by invoking remote services through a proxy.

How does it work?

Circuit breaker is a proxy element between two microservices which may appears in three states: Closed, Open, and Limited (Half Open).

In Closed state, the circuit breaker allows microservices to communicate as usual and monitor the number of failures occurring within the defined time period.

If the failure count exceeds the specified threshold value, the circuit breaker will move to the Open state. It will completely block the communication between microservices.

In the Half-Open state, the circuit breaker will allow a limited number of requests to reach article service. If those requests are successful, the circuit breaker will switch the state to Closed and allow normal operations. If not, it will again block the requests for the defined timeout period.

Implementing with SDK

The circuit breaker pattern is very popular in microservice communication and implemented in all of variety of development platforms.

In .Net the recommended approach for circuit breakers is Polly.

Using Polly is straightforward. Just adding a circuit breaker policy into your IHttpClientFactory outgoing middleware pipeline is as simple as adding a single incremental piece of code to what you already have when using IHttpClientFactory. The only addition here to the code used for HTTP call retries is the code where you add the Circuit Breaker policy to the list of policies to use, as shown in the following incremental code:

// Program.cs
var retryPolicy = GetRetryPolicy();
var circuitBreakerPolicy = GetCircuitBreakerPolicy();

builder.Services.AddHttpClient<IBasketService, BasketService>()
        .SetHandlerLifetime(TimeSpan.FromMinutes(5))  // Sample: default lifetime is 2 minutes
        .AddHttpMessageHandler<HttpClientAuthorizationDelegatingHandler>()
        .AddPolicyHandler(retryPolicy)
        .AddPolicyHandler(circuitBreakerPolicy);

...
static IAsyncPolicy<HttpResponseMessage> GetCircuitBreakerPolicy()
{
    return HttpPolicyExtensions
        .HandleTransientHttpError()
        .CircuitBreakerAsync(5, TimeSpan.FromSeconds(30));
}

Implementing with NGINX load-balacing pool

NGINX is a very popular event-based proxy server which commonly used to communicate in microservice infrastructure.

Circuit breaker pattern improves consistency and provides flexibility:

If a resizer instance indicates that it is unhealthy, the uploader routes requests to other instances but keeps checking to see whether that resizer instance has recovered. When the resizer instance indicates it is healthy again, it is put back into the load‑balanced pool and the uploader slowly ramps traffic up to the instance’s full capacity. This design prevents instances of the resizer from failing completely, prevents work from being begun but not completed, prevents excessive waiting for users whose processes would otherwise have failed, and helps the system deal most effectively with the request stream sent to it.

Circuit breaking in Kubernetes with Istio and Invoy proxy

Service mesh pattern is commonly used in containerized architecture to encapsulate network connectivity and provide observability in Kubernetes cluster.

In Istio service mesh the circuit breaking are implemented using Envoy proxy configuration, which might be done in a few variations:

• Cluster maximum connections: The maximum number of connections that Envoy will establish to all hosts in an upstream cluster. All connections, whether active or draining, count against this limit. Even if this circuit breaker has overflowed, Envoy will ensure that a host selected by cluster load balancing has at least one connection allocated.
• Cluster maximum pending requests: The maximum number of requests that will be queued while waiting for a ready connection pool connection. Requests are added to the list of pending requests whenever there aren’t enough upstream connections available to immediately dispatch the request.
• Cluster maximum requests: The maximum number of requests that can be outstanding to all hosts in a cluster at any given time.
• Cluster maximum active retries: The maximum number of retries that can be outstanding to all hosts in a cluster at any given time.
• Cluster maximum concurrent connection pools: The maximum number of connection pools that can be concurrently instantiated. When a cluster has exhausted its concurrent connection pools, it will attempt to reclaim an idle one. If it cannot, then the circuit breaker will overflow.

To configurate the circuit breaker:

$ kubectl apply -f - <<EOF
apiVersion: networking.istio.io/v1alpha3
kind: DestinationRule
metadata:
  name: httpbin
spec:
  host: httpbin
  trafficPolicy:
    connectionPool:
      tcp:
        maxConnections: 1
      http:
        http1MaxPendingRequests: 1
        maxRequestsPerConnection: 1
    outlierDetection:
      consecutive5xxErrors: 1
      interval: 1s
      baseEjectionTime: 3m
      maxEjectionPercent: 100
EOF

Pros and Cons

Pros	Cons
Reliability: helps to prevent cascading failures	Need good infrastructure management to maintain circuit breakers
Error handling: handles errors gracefully and provides better under experience	Throughput issues in services if not properly configured.
Reduces the application downtimes	Quite tricky to test
Asynchronous: suitable for handling asynchronous communications
Enhanced observability: state changes of the circuit breaker can be used for error monitoring
Easy to install and use: variety of SDK solutions, infrastructure integration, Kubernetes implementations available.
Integrity: easy integrated into existed infrastructure load-balancing solution

Hope you like our new architecture patterns seria. For more content please subscribe to our newsletter, follow us on Twitter and LinkedIn.

Save your privacy, bean ethical!