Yet another Camel Circuit Breaker…

Camel 2.19 was recently released. You can read about that on Claus Ibsen’s blog. A quick look through the release notes and it will be readily apparent that there are numerous new features that went into this version.

452246813_fd26cb6539_z.jpg
Camel: modified from original

One of the new features is the ThrottlingExceptionRoutePolicy, which implements the circuit breaker pattern.

The circuit breaker pattern is described by Martin Fowler on his bliki as follows:

The basic idea behind the circuit breaker is very simple. You wrap a protected function call in a circuit breaker object, which monitors for failures. Once the failures reach a certain threshold, the circuit breaker trips, and all further calls to the circuit breaker return with an error, without the protected call being made at all.

Now if you have used Camel for any length of time you may be wondering why is there another implementation of this pattern. After all Camel already provides two.

The first was released with 2.14 and was added by extending LoadBalancerSupport in the CircuitBreakerLoadBalancer.  The documentation can be found on the Camel site under the Load Balancer EIP. The slightly modified sample DSL is from the Camel documentation.

from("direct:start")
    .onException(RejectedExecutionException.class)
        .handled(true)
        .transform().constant("service unavailable")
    .end()
    .loadBalance()
          .circuitBreaker(2, 1000L, MyCustomException.class)
          .to("mock:service")
    .end();

In the prior release of Camel (2.18), the Netflix Hystrix circuit breaker implementation was added to the expanding set of Camel capabilities. The slightly modified sample DSL is from the Camel documentation.

from("direct:start")
    .hystrix()
        .to("mock:service")
    .onFallback()
        .transform().constant("service unavailable")
    .end()

Both of these options are used to protect the consumer of a Camel route from external resources that are not responding or exceed some specified timeout threshold. The idea is that once the specified number of failures has occurred the route will stop calling mock:service and instead perform some faster operation. In this case return “service unavailable”.

But what happens if we would like to stop consuming from the endpoint when the circuit is in the open state rather than bypass the service call. This is what the ThrottlingExceptionRoutePolicy provides. It is based on the CircuitBreakerLoadBalancer and is implemented as a Camel RoutePolicy.

Here is how it basically works. All un-handled exceptions thrown from the route are passed to the RoutePolicy where they are evaluated. Based on the settings the circuit breaker will count the number of exceptions (failureThreshold) over a period of time (failureWindow). If the threshold is met in the time allotted then the circuit is opened and the route will stop consuming from the endpoint. This might make sense when one is consuming from a JMS queue or a directory with files. Rather than read a message and roll it back continually, the route can stop using resources for a period of time when the external services it depends on are failing. 

Here is a sample DSL:

onException(BadServiceException.class)
    .handled(false); // open circuit

onException(SomethingElseException.class)
    .handled(true); // don't open circuit

from("jms:queue:start")
    .routePolicy(circuitPolicy)
    .to("mock:service")

Assuming one is using Spring, this is how the RoutePolicy would be configured. The fourth parameter is a List that can be used to limit the Exceptions that should be used to determine whether to open a circuit.

@Bean
public ThrottlingExceptionRoutePolicy circuitPolicy() {
    return new ThrottlingExceptionRoutePolicy(
        this.getFailureThreshold(),
        this.getFailureWindow(),
        this.getHalfOpenAfter(), null);
}

After a specified amount of time (halfOpenAfter) the circuit will move to the half open state and start to consume messages from the endpoint. If any failures occur during the half open state the circuit will return to the open state. If messages are successfully processed it will move to the closed state.

Since the route is started in the half open state it is possible that several messages might be consumed from the endpoint. To avoid this situation the ThrottingExceptionHalfOpenHandlerinterface is provided.

public interface ThrottingExceptionHalfOpenHandler {
    boolean isReadyToBeClosed();
}

If an implementation of this interface is available it will be used instead of starting the route. Implementations can be used to check the external resources to see if they are available. For example they might issue a simple SQL statement

select 1 from dual

or call the REST service and evaluate the response.

Advertisements

One thought on “Yet another Camel Circuit Breaker…

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s