Spring Cloud Gateway Hystrix

In the microservice architecture, if the downstream dependency problem occurs, if the upstream caller does not do request degradation processing, the downstream abnormal dependency is not isolated, it is likely that one or two services or as small as one or two interface abnormalities lead to the unavailability of all upstream services, or even affect the whole business line. The mainstream request degradation processing is still the Hystrix from Netfilx. The working principle of Hystrix is.

Isolates requests based on thread pools or semaphores, and once a downstream service fails to respond within a specified configured timeout it enters a pre-defined or default degraded implementation.
The status of each request is logged, and the rate of processing failures within a sliding window exceeding a set threshold triggers a Circle Breaker to open, after which all requests go directly to the preset or default degradation logic.
After the fuse is opened and the time since the fuse was opened or the time since the last trial request was released exceeds a set value, the fuse breaker enters a half-open state and allows the release of a trial request.
After the request success rate increases, the fuse is closed based on statistical data determination and all requests are released normally.

Instead of going into the details of Hystrix, let’s move on to how to use Hystrix in Spring Cloud Gateway, mainly including the built-in Hystrix filters and custom filters combined with Hystrix to achieve the functionality we want. In addition to introducing the spring-cloud-starter-gateway dependency, you also need to introduce spring-cloud-starter-netflix-hystrix.

<dependencies>
    <dependency>
        <groupId>org.springframework.cloud</groupId>
        <artifactId>spring-cloud-starter-gateway</artifactId>
    </dependency>
    <dependency>
        <groupId>org.springframework.cloud</groupId>
        <artifactId>spring-cloud-starter-netflix-hystrix</artifactId>
    </dependency>
</dependencies>

Use the built-in Hystrix filter

The built-in Hystrix filter is the HystrixGatewayFilterFactory, which supports the following configurations.

public static class Config {
    // If the following Setter is configured as null, name will be used as the HystrixCommandKey of the Hystrix
    private String name;
    // Setter property of Hystrix, mainly used to configure the KEY and other properties of the command
    private Setter setter;
    // The target URI for degradation must start with forward, and the URI will match the controller method applied to the gateway
    private URI fallbackUri;

	public String getName() {
		return name;
	}

	public Config setName(String name) {
		this.name = name;
		return this;
	}

    public Config setFallbackUri(String fallbackUri) {
	    if (fallbackUri != null) {
			setFallbackUri(URI.create(fallbackUri));
		}
		return this;
	}

    public URI getFallbackUri() {
	    return fallbackUri;
    }
    
    // Note that for this method, the configured fallbackUri should start with forward as the schema, otherwise it will throw an exception
    public void setFallbackUri(URI fallbackUri) {
        if (fallbackUri != null && !"forward".equals(fallbackUri.getScheme())) {
			throw new IllegalArgumentException("Hystrix Filter currently only supports 'forward' URIs, found " + fallbackUri);
		}
		this.fallbackUri = fallbackUri;
	}

	public Config setSetter(Setter setter) {
		this.setter = setter;
		return this;
	}
}

In addition,

(1) the global Hystrix configuration will also take effect for HystrixGatewayFilterFactory;

(2) HystrixGatewayFilterFactory can be used as default-filters for all routing configurations as under-the-hood filters and function as such.

For point (1), if we configure the following in application.yaml.

spring:
  cloud:
    gateway:
      routes:
        - id: order_route
          uri: http://localhost:9091
          predicates:
            - Path=/order/**
      default-filters:
        - name: Hystrix
          args:
            name: HystrixCommand
            fallbackUri: forward:/fallback

When I was testing, I found that the Setter mentioned above could not be configured, presumably because the Setter object of Hystrix is multi-packaged and there is no way to set the property for the time being. Next we have to add a controller method to the gateway service for handling redirected /fallback requests:

@RestController
public class FallbackController {

    @RequestMapping(value = "/fallback")
    @ResponseStatus
    public Mono<Map<String, Object>> fallback(ServerWebExchange exchange, Throwable throwable) {
        Map<String, Object> result = new HashMap<>(8);
        ServerHttpRequest request = exchange.getRequest();
        result.put("path", request.getPath().pathWithinApplication().value());
        result.put("method", request.getMethodValue());
        if (null != throwable.getCause()) {
            result.put("message", throwable.getCause().getMessage());
        } else {
            result.put("message", throwable.getMessage());
        }
        return Mono.just(result);
    }
}

Controller method entries are handled by the internal components of Spring Cloud Gateway and can call back some useful types such as ServerWebExchange instances, specific exception instances and so on.

Custom Filters with Hystrix

The HystrixGatewayFilterFactory should meet business needs in most cases, but here also do a customization of a filter that integrates Hystrix and implements the following functionality.

Create a new instance of the Hystrix command to be invoked based on each request URL.
Each URL can specify a unique thread pool configuration, or use the default if not specified.
A separate Hystrix timeout can be configured for each URL.

That is, each different external request URL is isolated using a thread pool via Hystrix. Of course, such a filter only makes sense if the number of different URLs for external requests is limited, otherwise there is a risk of creating too many thread pools causing system performance degradation, which is counterproductive. The modification is as follows.

@Component
public class CustomHystrixFilter extends AbstractGatewayFilterFactory<CustomHystrixFilter.Config> {

    private static final String FORWARD_KEY = "forward";
    private static final String NAME = "CustomHystrix";
    private static final int TIMEOUT_MS = 1000;
    private final ObjectProvider<DispatcherHandler> dispatcherHandlerProvider;
    private volatile DispatcherHandler dispatcherHandler;
    private boolean processConfig = false;

    public CustomHystrixFilter(ObjectProvider<DispatcherHandler> dispatcherHandlerProvider) {
        super(Config.class);
        this.dispatcherHandlerProvider = dispatcherHandlerProvider;
    }

    private DispatcherHandler getDispatcherHandler() {
        if (dispatcherHandler == null) {
            dispatcherHandler = dispatcherHandlerProvider.getIfAvailable();
        }

        return dispatcherHandler;
    }

    @Override
    public List<String> shortcutFieldOrder() {
        return Collections.singletonList(NAME_KEY);
    }


    @Override
    public GatewayFilter apply(Config config) {
        processConfig(config);
        return (exchange, chain) -> {
            ServerHttpRequest request = exchange.getRequest();
            String path = request.getPath().pathWithinApplication().value();
            int timeout = config.getTimeout().getOrDefault(path, TIMEOUT_MS);
            CustomHystrixCommand command = new CustomHystrixCommand(config.getFallbackUri(), exchange, chain, timeout, path);
            return Mono.create(s -> {
                Subscription sub = command.toObservable().subscribe(s::success, s::error, s::success);
                s.onCancel(sub::unsubscribe);
            }).onErrorResume((Function<Throwable, Mono<Void>>) throwable -> {
                if (throwable instanceof HystrixRuntimeException) {
                    HystrixRuntimeException e = (HystrixRuntimeException) throwable;
                    HystrixRuntimeException.FailureType failureType = e.getFailureType();
                    switch (failureType) {
                        case TIMEOUT:
                            return Mono.error(new TimeoutException());
                        case COMMAND_EXCEPTION: {
                            Throwable cause = e.getCause();
                            if (cause instanceof ResponseStatusException || AnnotatedElementUtils
                                    .findMergedAnnotation(cause.getClass(), ResponseStatus.class) != null) {
                                return Mono.error(cause);
                            }
                        }
                        default:
                            break;
                    }
                }
                return Mono.error(throwable);
            }).then();
        };
    }

    /**
     * YAML解析的时候MAP的KEY不支持'/'，这里只能用'-'替代
     *
     * @param config config
     */
    private void processConfig(Config config) {
        if (!processConfig) {
            processConfig = true;
            if (null != config.getTimeout()) {
                Map<String, Integer> timeout = new HashMap<>(8);
                config.getTimeout().forEach((k, v) -> {
                    String key = k.replace("-", "/");
                    if (!key.startsWith("/")) {
                        key = "/" + key;
                    }
                    timeout.put(key, v);
                });
                config.setTimeout(timeout);
            }
        }
    }

    @Override
    public String name() {
        return NAME;
    }

    private class CustomHystrixCommand extends HystrixObservableCommand<Void> {

        private final URI fallbackUri;
        private final ServerWebExchange exchange;
        private final GatewayFilterChain chain;

        public CustomHystrixCommand(URI fallbackUri,
                                    ServerWebExchange exchange,
                                    GatewayFilterChain chain,
                                    int timeout,
                                    String key) {
            super(Setter.withGroupKey(HystrixCommandGroupKey.Factory.asKey(key))
                    .andCommandKey(HystrixCommandKey.Factory.asKey(key))
                    .andCommandPropertiesDefaults(HystrixCommandProperties.Setter().withExecutionTimeoutInMilliseconds(timeout)));
            this.fallbackUri = fallbackUri;
            this.exchange = exchange;
            this.chain = chain;
        }

        @Override
        protected Observable<Void> construct() {
            return RxReactiveStreams.toObservable(this.chain.filter(exchange));
        }

        @Override
        protected Observable<Void> resumeWithFallback() {
            if (null == fallbackUri) {
                return super.resumeWithFallback();
            }
            URI uri = exchange.getRequest().getURI();
            boolean encoded = containsEncodedParts(uri);
            URI requestUrl = UriComponentsBuilder.fromUri(uri)
                    .host(null)
                    .port(null)
                    .uri(this.fallbackUri)
                    .build(encoded)
                    .toUri();
            exchange.getAttributes().put(GATEWAY_REQUEST_URL_ATTR, requestUrl);
            ServerHttpRequest request = this.exchange.getRequest().mutate().uri(requestUrl).build();
            ServerWebExchange mutated = exchange.mutate().request(request).build();
            return RxReactiveStreams.toObservable(getDispatcherHandler().handle(mutated));
        }
    }

    public static class Config {

        private String id;
        private URI fallbackUri;
        /**
         * url -> timeout ms
         */
        private Map<String, Integer> timeout;

        public String getId() {
            return id;
        }

        public Config setId(String id) {
            this.id = id;
            return this;
        }

        public URI getFallbackUri() {
            return fallbackUri;
        }

        public Config setFallbackUri(URI fallbackUri) {
            if (fallbackUri != null && !FORWARD_KEY.equals(fallbackUri.getScheme())) {
                throw new IllegalArgumentException("Hystrix Filter currently only supports 'forward' URIs, found " + fallbackUri);
            }
            this.fallbackUri = fallbackUri;
            return this;
        }

        public Map<String, Integer> getTimeout() {
            return timeout;
        }

        public Config setTimeout(Map<String, Integer> timeout) {
            this.timeout = timeout;
            return this;
        }
    }
}

In fact, most of the code is similar to the built-in Hystrix filter, only the command transformation function part and the configuration loading processing part have been changed. The configuration file is as follows.

spring:
  cloud:
    gateway:
      routes:
        - id: hystrix_route
          uri: http://localhost:9091
          predicates:
            - Host=localhost:9090
          filters:
            - name: CustomHystrix
              args:
                id: CustomHystrix
                fallbackUri: forward:/fallback
                timeout:
                  # URLs are separated by - here for now, because / does not support
                  order-remote: 2000
  application:
    name: route-server
server:
  port: 9090

The gateway adds a /fallback processing controller as follows.

@RestController
public class FallbackController {

    @RequestMapping(value = "/fallback")
    @ResponseStatus
    public Mono<Map<String, Object>> fallback(ServerWebExchange exchange, Throwable throwable) {
        Map<String, Object> result = new HashMap<>(8);
        ServerHttpRequest request = exchange.getRequest();
        result.put("path", request.getPath().pathWithinApplication().value());
        result.put("method", request.getMethodValue());
        if (null != throwable.getCause()) {
            result.put("message", throwable.getCause().getMessage());
        } else {
            result.put("message", throwable.getMessage());
        }
        return Mono.just(result);
    }
}

Intentional downstream service interruption points.

curl http://localhost:9090/order/remote


Response：
{
    "path": "/fallback",
    "method": "GET",
    "message": null   # <== Here the message is null because it is a timeout exception
}

Just in line with the expected results.

Summary

This article is just to provide a usable example and problem solving ideas for Hystrix and filter applications.

The specific use of Hystrix still needs to be considered for real world scenarios.

Reference https://www.throwx.cn/2019/05/25/spring-cloud-gateway-hystrix/

Table of Contents

Use the built-in Hystrix filter

Custom Filters with Hystrix

Summary