Spring Boot logs access logs in several ways

Certain business requirements require tracking our interface access, i.e. logging the requests and responses. The basic logging dimension contains request parameters (path query parameters, request body), request path (uri), request method (method), request headers (headers), and response status, response headers, and even contains sensitive response bodies, etc. Today summarizes several methods, you can choose as needed.

How request tracking is implemented

Gateways

Many gateway facilities have httptrace capabilities that help us centrally log request traffic. orange, Kong, Apache Apisix, all Nginx-based gateways have this capability, and even Nginx itself provides the ability to log httptrace logs.

Advantages are that httptrace logs can be managed centrally and development-free; Disadvantages are that they are technically demanding and require supporting distribution, storage, and query facilities.

Spring Boot Actuator

In Spring Boot, a simple tracing feature is actually provided. You just need to integrate.

<dependency>
    <groupId>org.springframework.boot</groupId>
    <artifactId>spring-boot-starter-actuator</artifactId>
</dependency>

Enable /actuator/httptrace.

management:
  endpoints:
    web:
      exposure:
        include: 'httptrace'

It will be possible to get information about recent Http requests via http://server:port/actuator/httptrace.

However, in the latest versions it may be necessary to explicitly declare how this trace information is stored, i.e. by implementing the HttpTraceRepository interface and injecting Spring IoC.

For example in memory and limited to the last 100 (not recommended for production use).

@Bean
public HttpTraceRepository httpTraceRepository(){
    return new InMemoryHttpTraceRepository();
}

The trace log is presented in json format.

springcloud

Not many dimensions are logged, but you can certainly try it if it’s enough.

Pros in that it’s easy to integrate and almost development free; Cons in that it doesn’t log many dimensions and you need to build a facility to cache and consume these log messages.

CommonsRequestLoggingFilter

The Spring Web module also provides a filter CommonsRequestLoggingFilter that logs the request details. It is also relatively simple to configure.

@Bean
CommonsRequestLoggingFilter  loggingFilter(){
    CommonsRequestLoggingFilter loggingFilter = new CommonsRequestLoggingFilter();
    // 记录 客户端 IP信息
    loggingFilter.setIncludeClientInfo(true);
    // 记录请求头
    loggingFilter.setIncludeHeaders(true);
    // 如果记录请求头的话，可以指定哪些记录，哪些不记录
    // loggingFilter.setHeaderPredicate();
    // 记录 请求体  特别是POST请求的body参数
    loggingFilter.setIncludePayload(true);
    // 请求体的大小限制 默认50
    loggingFilter.setMaxPayloadLength(10000);
    //记录请求路径中的query参数 
    loggingFilter.setIncludeQueryString(true);
    return loggingFilter;
}

And the debug logging for CommonsRequestLoggingFilter must be turned on.

logging:
  level:
    org:
      springframework:
        web:
          filter:
            CommonsRequestLoggingFilter: debug

A request outputs the log twice, once before the first pass through the filter; and once after completing the filter chain.

springcloud

Here’s one more thing that can actually be converted to output json format.

The advantage is the flexibility of configuration and the comprehensive dimensionality of request tracking, disadvantage is that only the request is logged and not the response.

ResponseBodyAdvice

The Spring Boot unified return body can actually be logged, but you need to implement it yourself. This is based on the CommonsRequestLoggingFilter method of parsing requests. The response body can also be obtained, but the response header and state are not clear because the life cycle is not clear, so it is not clear whether it is appropriate to obtain here, but it is an idea.

/**
 * @author felord.cn
 * @since 1.0.8.RELEASE
 */
@Slf4j
@RestControllerAdvice(basePackages = {"cn.felord.logging"})
public class RestBodyAdvice implements ResponseBodyAdvice<Object> {
    private static final int DEFAULT_MAX_PAYLOAD_LENGTH = 10000;
    public static final String REQUEST_MESSAGE_PREFIX = "Request [";
    public static final String REQUEST_MESSAGE_SUFFIX = "]";
    private ObjectMapper objectMapper = new ObjectMapper();

    @Override
    public boolean supports(MethodParameter returnType, Class<? extends HttpMessageConverter<?>> converterType) {
        return true;
    }

    @SneakyThrows
    @Override
    public Object beforeBodyWrite(Object body,
                                  MethodParameter returnType,
                                  MediaType selectedContentType,
                                  Class<? extends HttpMessageConverter<?>> selectedConverterType,
                                  ServerHttpRequest request,
                                  ServerHttpResponse response) {

        ServletServerHttpRequest servletServerHttpRequest = (ServletServerHttpRequest) request;

        log.debug(createRequestMessage(servletServerHttpRequest.getServletRequest(), REQUEST_MESSAGE_PREFIX, REQUEST_MESSAGE_SUFFIX));
        Rest<Object> objectRest;
        if (body == null) {
            objectRest = RestBody.okData(Collections.emptyMap());
        } else if (Rest.class.isAssignableFrom(body.getClass())) {
            objectRest = (Rest<Object>) body;
        }
        else if (checkPrimitive(body)) {
            return RestBody.okData(Collections.singletonMap("result", body));
        }else {
            objectRest = RestBody.okData(body);
        }
        log.debug("Response Body ["+ objectMapper.writeValueAsString(objectRest) +"]");
        return objectRest;
    }


    private boolean checkPrimitive(Object body) {
        Class<?> clazz = body.getClass();
        return clazz.isPrimitive()
                || clazz.isArray()
                || Collection.class.isAssignableFrom(clazz)
                || body instanceof Number
                || body instanceof Boolean
                || body instanceof Character
                || body instanceof String;
    }


    protected String createRequestMessage(HttpServletRequest request, String prefix, String suffix) {
        StringBuilder msg = new StringBuilder();
        msg.append(prefix);
        msg.append(request.getMethod()).append(" ");
        msg.append(request.getRequestURI());


        String queryString = request.getQueryString();
        if (queryString != null) {
            msg.append('?').append(queryString);
        }


        String client = request.getRemoteAddr();
        if (StringUtils.hasLength(client)) {
            msg.append(", client=").append(client);
        }
        HttpSession session = request.getSession(false);
        if (session != null) {
            msg.append(", session=").append(session.getId());
        }
        String user = request.getRemoteUser();
        if (user != null) {
            msg.append(", user=").append(user);
        }

        HttpHeaders headers = new ServletServerHttpRequest(request).getHeaders();
        msg.append(", headers=").append(headers);

        String payload = getMessagePayload(request);
        if (payload != null) {
            msg.append(", payload=").append(payload);
        }

        msg.append(suffix);
        return msg.toString();
    }

    protected String getMessagePayload(HttpServletRequest request) {
        ContentCachingRequestWrapper wrapper =
                WebUtils.getNativeRequest(request, ContentCachingRequestWrapper.class);
        if (wrapper != null) {
            byte[] buf = wrapper.getContentAsByteArray();
            if (buf.length > 0) {
                int length = Math.min(buf.length, DEFAULT_MAX_PAYLOAD_LENGTH);
                try {
                    return new String(buf, 0, length, wrapper.getCharacterEncoding());
                } catch (UnsupportedEncodingException ex) {
                    return "[unknown]";
                }
            }
        }
        return null;
    }
}

Don’t forget to configure the logging level of ResponseBodyAdvice to be DEBUG.

logstash-logback-encoder

This is logstash’s logback encoder that structures the output of httptrace to json. import.

<dependency>
    <groupId>net.logstash.logback</groupId>
    <artifactId>logstash-logback-encoder</artifactId>
    <version>6.6</version>
</dependency>

Add a ConsoleAppender to the logback configuration as LogstashEncoder :

<configuration>
    <appender name="jsonConsoleAppender" class="ch.qos.logback.core.ConsoleAppender">
        <encoder class="net.logstash.logback.encoder.LogstashEncoder"/>
    </appender>
    <root level=" INFO">
        <appender-ref ref="jsonConsoleAppender"/>
    </root>
</configuration>

Then also implement a parsing Filter :

import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import org.slf4j.MDC;
import org.springframework.core.annotation.Order;
import org.springframework.stereotype.Component;

import javax.servlet.*;
import javax.servlet.http.HttpServletRequest;
import javax.servlet.http.HttpServletResponse;
import java.io.IOException;
import java.util.UUID;

/**
 * @author felord.cn
 * @since 1.0.8.RELEASE
 */
@Order(1)
@Component
public class MDCFilter implements Filter {

    private final Logger LOGGER = LoggerFactory.getLogger(MDCFilter.class);
    private final String X_REQUEST_ID = "X-Request-ID";

    @Override
    public void doFilter(ServletRequest request,
                         ServletResponse response,
                         FilterChain chain) throws IOException, ServletException {
        HttpServletRequest req = (HttpServletRequest) request;
        HttpServletResponse res = (HttpServletResponse) response;
        try {
            addXRequestId(req);
            LOGGER.info("path: {}, method: {}, query {}",
                    req.getRequestURI(), req.getMethod(), req.getQueryString());
            res.setHeader(X_REQUEST_ID, MDC.get(X_REQUEST_ID));
            chain.doFilter(request, response);
        } finally {
            LOGGER.info("statusCode {}, path: {}, method: {}, query {}",
                    res.getStatus(), req.getRequestURI(), req.getMethod(), req.getQueryString());
            MDC.clear();
        }
    }

    private void addXRequestId(HttpServletRequest request) {
        String xRequestId = request.getHeader(X_REQUEST_ID);
        if (xRequestId == null) {
            MDC.put(X_REQUEST_ID, UUID.randomUUID().toString());
        } else {
            MDC.put(X_REQUEST_ID, xRequestId);
        }
    }

}

Here the parsing method can actually be more refined.

Not only can the interface request logs be recorded, but they can also be structured as json.

{"@timestamp":"2021-08-10T23:48:51.322+08:00","@version":"1","message":"statusCode 200, path: /log/get, method: GET, query foo=xxx&bar=ooo","logger_name":"cn.felord.logging.MDCFilter","thread_name":"http-nio-8080-exec-1","level":"INFO","level_value":20000,"X-Request-ID":"7c0db56c-b1f2-4d85-ad9a-7ead67660f96"}

Summary

Today introduced a number of ways to record the response to tracking interface requests, there is always one for you. Maybe you have a better way, welcome to share in the comments.

Reference https://felord.cn/mvc-httptrace.html

Table of Contents