打造一款适合自己的快速开发框架-http请求日志全局处理

1,923 阅读8分钟

前言

一般来说任何系统都会考虑请求日志的记录,一是为了方便记录用户的操作;二可能是为了后续的数据分析。而在springboot上,常用的收集方式有如下三种:过滤器Filter、拦截器Interceptor、AOP。这里就不对这三种方式进行比较了,感兴趣的同学可以去了解一下。

日志记录内容

做日志记录之前,肯定要先知道要记录什么,这里就将一些比较关键的信息罗列出来。

记录项 记录说明 记录来源
trackId 请求唯一标识,可以自定义规则生成 自定义生成策略,可以先使用uuid
uri 请求路径 request
queryString 请求url上的参数 request
method 请求方式,GET/POST等 request
description 操作说明 controller上的自定义注解
ip 请求ip,客户端ip地址 request
body 请求体,请求正文的内容 request或RequestBodyAdvice
token 请求token,登录用户token,登录状态下存在 request
userId 请求用户id,登录用户id,登录状态下存在 token中解析
returnData 返回结果,请求的结果 response或ResponseBodyAdvice
startTime 开始时间,调用controller前的时间,或响应开始的时间 埋点
endTime 结束时间,响应结果输出前,controller执行完成后的时间 埋点

调用链说明

暂时就不画图了,还是以箭头和文字来描述。

  • Filter-start
    • Interceptor-start
      • ControllerAdvice-start
        • Aspect-start
          • Controller-start
          • Controller-end
        • Aspect-end
      • ControllerAdvice-end
    • Interceptor-end
  • Filter-end

上面是完整的调用链,本文实现的时候并没有使用Filter和Aspect,因为Filter拦截的请求过多,有好些不是自己想要的,放行处理的时候并不是很方便。而Aspect需要写的代码略多,所以在该场景下并没有考虑使用。本文的调用链如下:

  • Interceptor-start
    • ControllerAdvice-start
      • Controller-start
      • Controller-end
    • ControllerAdvice-end
  • Interceptor-end

处理流程说明

在这里为了方便说明,先以代码一点点的贴上代码片段。

AuthInterceptor.java

上一篇的权限拦截类

public class AuthInterceptor implements HandlerInterceptor {
    public boolean preHandle(HttpServletRequest request,
                HttpServletResponse response, Object handler) throws Exception {
        // 在这里处理id/uri/queryString/method/ip/token/userId/startTime,并将其set入ThreadLocal
        System.err.println("Interceptor-start");
    }
    @Override
	public void afterCompletion(HttpServletRequest request,
			HttpServletResponse response, Object handler, Exception ex)
			throws Exception {
        // 在这里已经能拿拿到所有要记录的内容了,可以使用异步的方式写入日志文件或是入库,然后还得remove ThreadLocal
		System.err.println("Interceptor-end");
	}
}

GlobalRequestBodyAdvice.java

在这里获取请求body

@ControllerAdvice
public class GlobalRequestBodyAdvice implements RequestBodyAdvice{
	private final static String charset = "UTF-8";
	@Override
	public boolean supports(MethodParameter methodParameter, Type targetType,
			Class<? extends HttpMessageConverter<?>> converterType) {
        // 这里返回true才会执行beforeBodyRead
		return true;
	}
	@Override
	public Object handleEmptyBody(Object body, HttpInputMessage inputMessage, MethodParameter parameter,
			Type targetType, Class<? extends HttpMessageConverter<?>> converterType) {
		return body;
	}
	@Override
	public HttpInputMessage beforeBodyRead(HttpInputMessage inputMessage, MethodParameter parameter, Type targetType,
			Class<? extends HttpMessageConverter<?>> converterType) throws IOException {
		// 这里处理body,先get ThreadLocal,再set
        String body = IOUtils.toString(inputMessage.getBody(), charset);
		System.err.println("ControllerAdvice-start");
		// body只能读一次,这里读了,得重新返回一个新的
		return new HttpInputMessage() {
			@Override
			public HttpHeaders getHeaders() {
				return inputMessage.getHeaders();
			}
			@Override
			public InputStream getBody() throws IOException {
				return new ByteArrayInputStream(body.getBytes());
			}
		};
	}
	@Override
	public Object afterBodyRead(Object body, HttpInputMessage inputMessage, MethodParameter parameter, Type targetType,
			Class<? extends HttpMessageConverter<?>> converterType) {
		return body;
	}
}

GlobalResponseBodyAdvice.java

这里获取返回的结果


/**
 * 全局的响应处理,这里能拿到控制层的返回参数,然后可以对该参数进行加密处理再返回,或是日志记录
 * @author mldong
 *
 */
@ControllerAdvice
public class GlobalResponseBodyAdvice implements ResponseBodyAdvice<Object>{
	@Override
	public boolean supports(MethodParameter returnType,
			Class<? extends HttpMessageConverter<?>> converterType) {
        // 这里返回true才会执行beforeBodyWrite
		return true;
	}
	@Override
	public Object beforeBodyWrite(Object returnData, MethodParameter returnType,
			MediaType selectedContentType,
			Class<? extends HttpMessageConverter<?>> selectedConverterType,
			ServerHttpRequest request, ServerHttpResponse response) {
        // 这里处理returnData,先get ThreadLocal,再set
		System.err.println("ControllerAdvice-end");
		return returnData;
	}
}

处理流程中因为涉及到多个对象中共享变量,所以使用了ThreadLocal,这里就不多做介绍了, 不明白的同学可以去了解一下。值得一得的是,springmvc的线程是复用的,所以执行结束后要记得remove。

开始编码

目录结构

├── mldong-admin  管理端接口
	├── src/main/java
├── mldong-common  工具类及通用代码
	├── src/main/java
		├──	com.mldong.common
			├──	interceptor
				└──	AuthInterceptor.java
			├── logger
					├──	DefaultLoggerStoreImpl.java
					├──	ILoggerStore.java
					└──	LoggerModel.java
			├── web
				├──	GlobalRequestBodyAdvice.java
				├──	GlobalResponseBodyAdvice.java
				└──	RequestHolder.java
				
├── mldong-generator  代码生成器

核心文件说明

  • mldong-common/src/main/java/com/mldong/common/interceptor/AuthInterceptor.java

权限拦截器,新增日志参数记录

package com.mldong.common.interceptor;

import io.swagger.annotations.ApiOperation;

import java.util.UUID;

import javax.servlet.http.HttpServletRequest;
import javax.servlet.http.HttpServletResponse;

import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.stereotype.Component;
import org.springframework.web.method.HandlerMethod;
import org.springframework.web.servlet.HandlerInterceptor;

import com.mldong.common.access.AccessInitProcessor;
import com.mldong.common.annotation.AuthIgnore;
import com.mldong.common.base.constant.GlobalErrEnum;
import com.mldong.common.exception.BizException;
import com.mldong.common.logger.ILoggerStore;
import com.mldong.common.logger.LoggerModel;
import com.mldong.common.web.RequestHolder;
@Component
public class AuthInterceptor implements HandlerInterceptor {
	@Autowired
	private AuthInterceptorService authInterceptorService;
	@Autowired
	private ILoggerStore loggerStore;
	@Override
	public boolean preHandle(HttpServletRequest request,
			HttpServletResponse response, Object handler) throws Exception {
		if(handler.getClass().isAssignableFrom(HandlerMethod.class)) {
            // 在这里新增logger的记录
			LoggerModel loggerModel = new LoggerModel();
			RequestHolder.setLoggerModel(loggerModel);
			loggerModel.setStartTime(System.currentTimeMillis());
			String trackId = UUID.randomUUID().toString();
			loggerModel.setTrackId(trackId);
			String uri = request.getRequestURI();
			loggerModel.setUri(uri);
			String queryString = request.getQueryString();
			loggerModel.setQueryString(null==queryString?"":queryString);
			String method = request.getMethod();
			loggerModel.setMethod(method);
			String ip = RequestHolder.getIPAddress();
			loggerModel.setIp(ip);
			HandlerMethod handlerMethod = (HandlerMethod) handler;
			ApiOperation apiOperation =  handlerMethod.getMethodAnnotation(ApiOperation.class);
			if(null!=apiOperation) {
				loggerModel.setDescription(apiOperation.value());
			}
			AuthIgnore authIgnore = handlerMethod.getMethodAnnotation(AuthIgnore.class);
			if(null != authIgnore) {
				// 要忽略权限
				return true;
			}
			String token = RequestHolder.getToken();
			loggerModel.setToken(token);
			if("".equals(token)) {
				throw new BizException(GlobalErrEnum.GL99990401);
			}
			Long userId = authInterceptorService.getUserId(token);
			loggerModel.setUserId(userId);
			if(!authInterceptorService.verifyToken(token)) {
				// token校验不通过
				throw new BizException(GlobalErrEnum.GL99990401);
			}
			RequestHolder.setUserId(userId);
			String access = AccessInitProcessor.getAccess(apiOperation);
			if(null == access) {
				// 没有定义,直接放行
				return true;
			}
			if(!authInterceptorService.hasAuth(token, access)){
				// 无权限访问
				throw new BizException(GlobalErrEnum.GL99990403);
			}
		}
		return true;
	}
	@Override
	public void afterCompletion(HttpServletRequest request,
			HttpServletResponse response, Object handler, Exception ex)
			throws Exception {
		LoggerModel loggerModel = RequestHolder.getLoggerModel();
		if(null != loggerModel) {
			loggerModel.setEndTime(System.currentTimeMillis());
			loggerStore.save(loggerModel);
		}
		// 记得要移除!!!!!
		RequestHolder.removeLoggerModel();
		// 记得要移除!!!!!
		RequestHolder.removeUserId();
	}
}
  • mldong-common/src/main/java/com/mldong/common/logger/LoggerModel.java

日志记录实体

package com.mldong.common.logger;

import java.io.Serializable;

/**
 * 日志实体类
 * @author mldong
 *
 */
public class LoggerModel implements Serializable{

	/**
	 * 
	 */
	private static final long serialVersionUID = 4296799309713867875L;
	/**
	 * 请求唯一标识,可以自定义规则生成
	 */
	private String trackId;
	/**
	 * 请求路径
	 */
	private String uri;
	/**
	 * 请求url上的参数
	 */
	private String queryString;
	/**
	 * 请求方式,GET/POST等
	 */
	private String method;
	/**
	 * 操作说明
	 */
	private String description;
	/**
	 * 请求ip,客户端ip地址
	 */
	private String ip;
	/**
	 * 请求体,请求正文的内容
	 */
	private String body;
	/**
	 * 请求token,登录用户token,登录状态下存在
	 */
	private String token;
	/**
	 * 请求用户id,登录用户id,登录状态下存在
	 */
	private Long userId;
	/**
	 * 返回结果,请求的结果
	 */
	private String returnData;
	private long startTime;
	private long endTime;
	// 省略 get set
	
}

  • mldong-common/src/main/java/com/mldong/common/logger/ILoggerStore.java

日志存储接口,如果需要入库,需要由mldong-admin层去实现。

package com.mldong.common.logger;

/**
 * 日志存储接口
 * @author mldong
 *
 */
public interface ILoggerStore {
	/**
	 * 存储日志
	 * @param model
	 * @return
	 */
	public int save(LoggerModel model);
}
  • mldong-common/src/main/java/com/mldong/common/logger/DefaultLoggerStoreImpl.java

简单的日志存储实现->slf4j

package com.mldong.common.logger;

import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import org.springframework.stereotype.Component;

/**
 * 这里先简单的日志输出
 * @author mldong
 *
 */
@Component
public class DefaultLoggerStoreImpl implements ILoggerStore {
	private final static Logger LOGGER = LoggerFactory.getLogger(DefaultLoggerStoreImpl.class);
	@Override
	public int save(LoggerModel model) {
		LOGGER.info("{}\t{}\t{}\t{}\t{}\t{}\t{}\t{}\t{}\t{}\t{}\t{}",
				model.getTrackId(),
				model.getUri(),
				model.getQueryString(),
				model.getMethod(),
				model.getDescription(),
				model.getIp(),
				model.getBody(),
				model.getToken(),
				model.getUserId(),
				model.getReturnData(),
				model.getStartTime(),
				model.getEndTime());
		return 1;
	}
}
  • mldong-common/src/main/java/com/mldong/common/web/GlobalRequestBodyAdvice.java

全局请求处理,这里主要setBody

package com.mldong.common.web;

import java.io.ByteArrayInputStream;
import java.io.IOException;
import java.io.InputStream;
import java.lang.reflect.Type;

import org.apache.commons.io.IOUtils;
import org.springframework.core.MethodParameter;
import org.springframework.http.HttpHeaders;
import org.springframework.http.HttpInputMessage;
import org.springframework.http.converter.HttpMessageConverter;
import org.springframework.http.converter.json.MappingJackson2HttpMessageConverter;
import org.springframework.web.bind.annotation.ControllerAdvice;
import org.springframework.web.servlet.mvc.method.annotation.RequestBodyAdvice;

import com.mldong.common.logger.LoggerModel;
/**
 * 全局的请求处理,可以在这里对原始的参数进行解密,或者请求参数日志记录
 * @author mldong
 *
 */
@ControllerAdvice
public class GlobalRequestBodyAdvice implements RequestBodyAdvice{
	private final static String charset = "UTF-8";
	@Override
	public boolean supports(MethodParameter methodParameter, Type targetType,
			Class<? extends HttpMessageConverter<?>> converterType) {
		return converterType.isAssignableFrom(MappingJackson2HttpMessageConverter.class);
	}
 
	@Override
	public Object handleEmptyBody(Object body, HttpInputMessage inputMessage, MethodParameter parameter,
			Type targetType, Class<? extends HttpMessageConverter<?>> converterType) {
		return body;
	}
 
	@Override
	public HttpInputMessage beforeBodyRead(HttpInputMessage inputMessage, MethodParameter parameter, Type targetType,
			Class<? extends HttpMessageConverter<?>> converterType) throws IOException {
		LoggerModel loggerModel = RequestHolder.getLoggerModel();
		if(null != loggerModel) {
			String body = IOUtils.toString(inputMessage.getBody(), charset);
            // 设置请求正文,这里拿到的是InputStream的内容
			loggerModel.setBody(body);
			// InputStream只能读一次,这里读了,得重新返回一个新的
			return new HttpInputMessage() {
				@Override
				public HttpHeaders getHeaders() {
					return inputMessage.getHeaders();
				}
				
				@Override
				public InputStream getBody() throws IOException {
					return new ByteArrayInputStream(body.getBytes());
				}
			};
		}
		return inputMessage;
	}
	@Override
	public Object afterBodyRead(Object body, HttpInputMessage inputMessage, MethodParameter parameter, Type targetType,
			Class<? extends HttpMessageConverter<?>> converterType) {
		return body;
	}

}
  • mldong-common/src/main/java/com/mldong/common/web/GlobalResponseBodyAdvice.java

全局的返回处理,这里主要setReturnData

package com.mldong.common.web;

import org.springframework.core.MethodParameter;
import org.springframework.http.MediaType;
import org.springframework.http.converter.HttpMessageConverter;
import org.springframework.http.server.ServerHttpRequest;
import org.springframework.http.server.ServerHttpResponse;
import org.springframework.web.bind.annotation.ControllerAdvice;
import org.springframework.web.servlet.mvc.method.annotation.ResponseBodyAdvice;

import com.mldong.common.tool.JsonTool;
import com.mldong.common.logger.LoggerModel;
/**
 * 全局的响应处理,这里能拿到控制层的返回参数,然后可以对该参数进行加密处理再返回,或是日志记录
 * @author mldong
 *
 */
@ControllerAdvice
public class GlobalResponseBodyAdvice implements ResponseBodyAdvice<Object>{

	@Override
	public boolean supports(MethodParameter returnType,
			Class<? extends HttpMessageConverter<?>> converterType) {
		return true;
	}

	@Override
	public Object beforeBodyWrite(Object returnData, MethodParameter returnType,
			MediaType selectedContentType,
			Class<? extends HttpMessageConverter<?>> selectedConverterType,
			ServerHttpRequest request, ServerHttpResponse response) {
		LoggerModel loggerModel = RequestHolder.getLoggerModel();
		if(null != loggerModel) {
            // 设置返回结果,这里拿到的是controller方法的返回值
			loggerModel.setReturnData(null==returnData?"":JsonTool.toJson(returnData));
		}
		return returnData;
	}
}
  • mldong-common/src/main/java/com/mldong/common/web/RequestHolder.java

    /**
     * 
     */
    package com.mldong.common.web;
    
    import javax.servlet.http.HttpServletRequest;
    
    import org.apache.commons.lang3.StringUtils;
    import org.springframework.web.context.request.RequestContextHolder;
    import org.springframework.web.context.request.ServletRequestAttributes;
    
    import com.mldong.common.base.constant.CommonConstants;
    import com.mldong.common.web.logger.LoggerModel;
    
    /**
     * @author mldong
     *
     */
    public class RequestHolder {
    	private final static ThreadLocal<Long> requestHolderUserId = new ThreadLocal<>();
    	private final static ThreadLocal<LoggerModel> requestHolderLoggerModel = new ThreadLocal<>();
    
    	private RequestHolder() {
    	}
    
    	public static void setUserId(Long userId) {
    		requestHolderUserId.set(userId);
    	}
    	public static Long getUserId() {
    		return requestHolderUserId.get();
    	}
    	public static void removeUserId() {
    		requestHolderUserId.remove();
    	}
    
    	public static void setLoggerModel(LoggerModel loggerModel) {
    		requestHolderLoggerModel.set(loggerModel);
    	}
    	public static LoggerModel getLoggerModel() {
    		return requestHolderLoggerModel.get();
    	}
    	public static void removeLoggerModel() {
    		requestHolderLoggerModel.remove();
    	}
    	/**
    	 * 获取ip地址
    	 * @return
    	 */
        public static String getIPAddress() {
        	HttpServletRequest  request = ((ServletRequestAttributes) RequestContextHolder.getRequestAttributes()).getRequest();
    
            String ip = null;
            String ipAddresses = request.getHeader("X-Forwarded-For");
            if (ipAddresses == null || ipAddresses.length() == 0 || "unknown".equalsIgnoreCase(ipAddresses)) {
                ipAddresses = request.getHeader("Proxy-Client-IP");
            }
            if (ipAddresses == null || ipAddresses.length() == 0 || "unknown".equalsIgnoreCase(ipAddresses)) {
                ipAddresses = request.getHeader("WL-Proxy-Client-IP");
            }
            if (ipAddresses == null || ipAddresses.length() == 0 || "unknown".equalsIgnoreCase(ipAddresses)) {
                ipAddresses = request.getHeader("HTTP_CLIENT_IP");
            }
            if (ipAddresses == null || ipAddresses.length() == 0 || "unknown".equalsIgnoreCase(ipAddresses)) {
                ipAddresses = request.getHeader("X-Real-IP");
            }
            if (ipAddresses != null && ipAddresses.length() != 0) {
                ip = ipAddresses.split(",")[0];
            }
            if (ip == null || ip.length() == 0 || "unknown".equalsIgnoreCase(ipAddresses)) {
                ip = request.getRemoteAddr();
            }
            return ip;
        }
        /**
         * 获取当前token
         * @return
         */
        public static String getToken() {
        	HttpServletRequest  request = ((ServletRequestAttributes) RequestContextHolder.getRequestAttributes()).getRequest();
    		String token = "";
    		token = request.getHeader(CommonConstants.TOKEN);
    		if(StringUtils.isEmpty(token)) {
    			token = request.getParameter(CommonConstants.TOKEN);
    		}
    		return token;
    	}
    }
    

小结

本文拿到请求日志只做了简单的输出处理,没有做黑白名单,没有入库,也没有对敏感字段进行过滤,后续内容会继续对该模块进行优化处理。

项目源码地址

  • 后端

gitee.com/mldong/mldo…

  • 前端

gitee.com/mldong/mldo…

相关文章

打造一款适合自己的快速开发框架-先导篇

打造一款适合自己的快速开发框架-后端脚手架搭建

打造一款适合自己的快速开发框架-集成mapper

打造一款适合自己的快速开发框架-集成swaggerui和knife4j

打造一款适合自己的快速开发框架-通用类封装之统一结果返回、统一异常处理

打造一款适合自己的快速开发框架-业务错误码规范及实践

打造一款适合自己的快速开发框架-框架分层及CURD样例

打造一款适合自己的快速开发框架-mapper逻辑删除及枚举类型规范

打造一款适合自己的快速开发框架-数据校验之Hibernate Validator

打造一款适合自己的快速开发框架-代码生成器原理及实现

打造一款适合自己的快速开发框架-通用查询设计与实现

打造一款适合自己的快速开发框架-基于rbac的权限管理

打造一款适合自己的快速开发框架-登录与权限拦截