Kong Rate-limiting源码解析-1

江湖十一郎
729 阅读4分钟

1. Rate-limiting 简介

Rate-limiting是kong 生态的一款开源限流插件， 提供根据时间窗口内请求数量限流的一款插件，其限流策略支持Local、 Redis、cluster, 限流粒度支持： consumer, credential, ip, service, header, path几个级别的限流， 限流方式可以用于 Global, Router, Service 。

Docker-compose.yml文件如下：

version: "3.7"
services: 
  kong-database:
    image: postgres:9.6
    container_name: kong-database
    ports:
      - "5432:5432"
    environment:
      - POSTGRES_USER=kong
      - POSTGRES_DB=kong
      - POSTGRES_PASSWORD=kongpass
    networks:
      - kong-net
    volumes:
      - ./docker-data/postgresql/postgresql:/var/lib/postgresql/data
  kong-gateway:
    # 镜像版本，目前最新
    image: kong/kong-gateway:2.8.1.0-alpine
    container_name: kong-gateway
    environment:
    # 数据持久化方式，使用postgres数据库
     - "KONG_DATABASE=postgres"
    # 数据库容器名称,Kong连接数据时使用些名称
     - "KONG_PG_HOST=kong-database"
     - "KONG_PG_USER=kong"
     - "KONG_PG_PASSWORD=kongpass"
    # 数据库名称
     - "KONG_CASSANDRA_CONTACT_POINTS=kong-database"
    # 日志记录目录
     - "KONG_PROXY_ACCESS_LOG=/dev/stdout"
     - "KONG_ADMIN_ACCESS_LOG=/dev/stdout"
     - "KONG_PROXY_ERROR_LOG=/dev/stderr"
     - "KONG_ADMIN_ERROR_LOG=/dev/stderr"
    # 暴露的端口
     - "KONG_ADMIN_LISTEN=0.0.0.0:8001, 0.0.0.0:8444 ssl"
     - "KONG_ADMIN_GUI_URL=http://localhost:8002"
    ports:
     - 8000:8000
     - 8443:8443
     - 8001:8001
     - 8444:8444
     - 8002:8002
     - 8445:8445
     - 8003:8003
     - 8004:8004
    # 使用docker网络
    networks:
     - kong-net
    # 依赖数据库服务
    depends_on:
     - kong-database
# kong 管理界面
  konga:
    image: pantsel/konga
    container_name: konga
    environment:
     - "TOKEN_SECRET=51liveup.cn"
     # 初始化使用development， 去初始化数据表， 正式使用production
     - "NODE_ENV=development"
     - "DB_ADAPTER=postgres" 
     - "DB_HOST=kong-database"
     - "DB_PORT=5432"
     - "DB_USER=kong"
     - "DB_PASSWORD=kongpass"
     - "DB_DATABASE=konga-db"
    ports:
     - 8080:1337
    networks:
     - kong-net
    depends_on:
     - kong-database
networks:
  kong-net:
    external: true

此文件执行启动时会包kong数据库初始化异常,需要执行初始化脚本,注意版本匹配

docker run --rm --network=kong-net  -e "KONG_DATABASE=postgres"  -e "KONG_PG_HOST=kong-database"  -e "KONG_PG_PASSWORD=kongpass" kong/kong-gateway:2.8.1.0-alpine kong migrations bootstrap

具体环境部署会在后期文章中详细讲解

下面进入正题，

因本次再使用Rate-limiting 插件中， 需要根据具体某一path 实现限流，其path需要支持通配， 故使用了konga UI 界面配置了 根据path限流， 如下

image.png 在测试过程中发现并未达到想要的效果， 即 path匹配则限流， path 不匹配则不限流， 在多次测试中发现， 均使用了Ip 限流，分析起源码发现是插件本身逻辑的问题

插件源码

路径： /usr/local/share/lua/5.1/kong/plugins/rate-limiting

local function get_identifier(conf)
  local identifier
  
  if conf.limit_by == "service" then
    identifier = (kong.router.get_service() or
                  EMPTY).id
  elseif conf.limit_by == "consumer" then
    identifier = (kong.client.get_consumer() or
                  kong.client.get_credential() or
                  EMPTY).id

  elseif conf.limit_by == "credential" then
    identifier = (kong.client.get_credential() or
                  EMPTY).id

  elseif conf.limit_by == "header" then
    identifier = kong.request.get_header(conf.header_name)

  elseif conf.limit_by == "path" then
    local req_path = kong.request.get_path()
    if req_path == conf.path then
      identifier = req_path
    end
  return identifier or kong.client.get_forwarded_ip()
end

逻辑解析： Rate limiting 只执行时需要根据 rout_id, service_id, indentifier, period_date, period生成一个唯一的local_key进行标识（如下图），而从上面的代码发现 indentifier是根你的限流粒度紧密相关，

从上面的代码可以看出在path匹配判断中， 如果path 匹配，将使用req_path作为identifier去后续生成local-key, 但是如果path不匹配，将使用ip 作为identifier， 故此时自动回退到使用ip 限流

local get_local_key = function(conf, identifier, period, period_date)
  local service_id, route_id = get_service_and_route_ids(conf)

  return fmt("ratelimit:%s:%s:%s:%s:%s", route_id, service_id, identifier,
             period_date, period)
end

2. 改造插件：

  1. 由于我的部署方式采用的是docker 部署，故需要先将脚本文件挂载出来， 再修改，对docker-compose 配置文件 kong-gateway服务下添加挂载volumes，如下
volumes:
     - ./docker-data/kong/handler.lua:/usr/local/share/lua/5.1/kong/plugins/rate-limiting/handler.lua
  1. 修改handler.lua脚本如下：

local function get_identifier(conf)
  local identifier
  if conf.limit_by == "service" then
    identifier = (kong.router.get_service() or
                  EMPTY).id
  elseif conf.limit_by == "consumer" then
    identifier = (kong.client.get_consumer() or
                  kong.client.get_credential() or
                  EMPTY).id

  elseif conf.limit_by == "credential" then
    identifier = (kong.client.get_credential() or
                  EMPTY).id

  elseif conf.limit_by == "header" then
    identifier = kong.request.get_header(conf.header_name)

  elseif conf.limit_by == "path" then
    local req_path = kong.request.get_path()
    --此处可以添加通配处理逻辑，
    if req_path == conf.path then
      identifier = req_path
    end
  end
  kong.client.get_forwarded_ip())
  --去掉默认ip 限流配置，但此时如果path 不匹配 identifier 将会是空（nil）
  --return identifier or kong.client.get_forwarded_ip()
  return identifier
end
  1. 修改此方法如下 local function get_usage(conf, identifier, current_timestamp, limits)
local function get_usage(conf, identifier, current_timestamp, limits)
  local usage = {}
  local stop
  --添加identifier 为空时不进行限流计算（源码没有这个为空校验），在后续处理中，stop为空，将不会进行限流
  if identifier then
	  for period, limit in pairs(limits) do
		local current_usage, err = policies[conf.policy].usage(conf, identifier, period, current_timestamp)
		if err then
		  return nil, nil, err
		end

		-- What is the current usage for the configured limit name?
		local remaining = limit - current_usage

		-- Recording usage
		usage[period] = {
		  limit = limit,
		  remaining = remaining,
		}

		if remaining <= 0 then
		  stop = period
		end
	  end
	end
  return usage, stop
end
  1. 主方法处理逻辑如下：

function RateLimitingHandler:access(conf)
  local current_timestamp = time() * 1000

  -- Consumer is identified by ip address or authenticated_credential id
  local identifier = get_identifier(conf)
  local fault_tolerant = conf.fault_tolerant

  -- Load current metric for configured period
  local limits = {
    second = conf.second,
    minute = conf.minute,
    hour = conf.hour,
    day = conf.day,
    month = conf.month,
    year = conf.year,
  }

  local usage, stop, err = get_usage(conf, identifier, current_timestamp, limits)
  if err then
    if not fault_tolerant then
      return error(err)
    end

    kong.log.err("failed to get usage: ", tostring(err))
  end
   --将限流信息放进header 中包含信息有: RATELIMIT_LIMIT, RATELIMIT_REMAINING,RATELIMIT_RESET等字段，可以在浏览器端查看
  if usage then
    -- Adding headers
    local reset
    local headers
    if not conf.hide_client_headers then
      headers = {}
      local timestamps
      local limit
      local window
      local remaining
      for k, v in pairs(usage) do
        local current_limit = v.limit
        local current_window = EXPIRATION[k]
        local current_remaining = v.remaining
        if stop == nil or stop == k then
          current_remaining = current_remaining - 1
        end
        current_remaining = max(0, current_remaining)

        if not limit or (current_remaining < remaining)
                     or (current_remaining == remaining and
                         current_window > window)
        then
          limit = current_limit
          window = current_window
          remaining = current_remaining

          if not timestamps then
            timestamps = timestamp.get_timestamps(current_timestamp)
          end

          reset = max(1, window - floor((current_timestamp - timestamps[k]) / 1000))
        end

        headers[X_RATELIMIT_LIMIT[k]] = current_limit
        headers[X_RATELIMIT_REMAINING[k]] = current_remaining
      end

      headers[RATELIMIT_LIMIT] = limit
      headers[RATELIMIT_REMAINING] = remaining
      headers[RATELIMIT_RESET] = reset
    end

    -- If limit is exceeded, terminate the request
    -- 如果此处stop 为空将不会进行限流，
    if stop then
      headers = headers or {}
      headers[RETRY_AFTER] = reset
      return kong.response.error(429, "API rate limit exceeded", headers)
    end

    if headers then
      kong.response.set_headers(headers)
    end
  end

  local ok, err = timer_at(0, increment, conf, limits, identifier, current_timestamp, 1)
  if not ok then
    kong.log.err("failed to create timer: ", err)
  end
end
avatar
文章
阅读
粉丝