开源API网关Orange源码分析

首先，了解一下Orange，Orange 是一个基于 OpenResty 的API网关。除 Nginx 的基本功能外，它还可用于API监控、访问控制(鉴权、WAF)、流量筛选、访问限速、AB测试、静/动态分流 等。 ​

说句实在的，它已经实现了绝大部分的功能，只不过目前已经处于停滞状态了

项目目录结构

• api
  • 目测是提供的接口，官方文档中提到了该项目提供了API接口用于实现第三方服务
• bin
  • 应该是运行目录，里面主要是lua的一些第三方包
• conf
  • 配置模板，类比nginx中的配置
• dashboard
  • 控制台程序
• docs
  • 运行文档
• install
  • 安装方法，包含了sql语句和安装执行脚本
• orange
  • 也是一些Lua的代码，主要是核心代码所在地
• rockspec
  • 似乎是对该项目的打包，类似于java的jar一样，相当于给别人直接使用
• test
  • 测试库

​

代码分析

启动Orange

初始化

先贴代码再分析

-- 执行过程:
-- 加载配置
-- 实例化存储store
-- 加载插件
-- 插件排序
function Orange.init(options)
    options = options or {}
    local store, config
    local status, err = pcall(function()
        local conf_file_path = options.config
        config = config_loader.load(conf_file_path)
        store = require("orange.store.mysql_store")(config.store_mysql)

        loaded_plugins = load_node_plugins(config, store)
        ngx.update_time()
        config.orange_start_at = ngx.now()
    end)

    if not status or err then
        ngx.log(ngx.ERR, "Startup error: " .. err)
        os.exit(1)
    end

    local consul = require("orange.plugins.consul_balancer.consul_balancer")
    consul.set_shared_dict_name("consul_upstream", "consul_upstream_watch")
    Orange.data = {
        store = store,
        config = config,
        consul = consul
    }

    -- init dns_client
    assert(dns_client.init())

    return config, store
end

不得不感叹，lua里面的匿名函数用得可针对，将方法直接作为函数的参数的方式，对编译器的词法分析器和语法分析器带来的挑战吧，有机会一定要拜读一下Lua的编译器。 首先，读取mysql的配置，然后加载配置的所有插件。 ​

其余的内容，就是定义的插件的基本方法的调用

插件定义的关键代码

对于插件，一共有下面的几种方法

• redirect()
• rewrite()
• access()
• balance()
• header_filter()
• body_filter()
• log()

也就是重定向，重写，接收（正常处理），平衡，请求响应头处理，内容处理以及日志处理。 ​

rewrite

redirect方法，默认的实现为：

function BasePlugin:rewrite()
    ngx.log(ngx.DEBUG, " executing plugin \"", self._name, "\": rewrite")
end

只是用ngx.log打印出了一个日志。 我们随便拎一个实现了该方法的插件子类来看看，这里以headers插件为例：

function HeaderHandler:rewrite(conf)
    HeaderHandler.super.rewrite(self)

    local enable = orange_db.get("headers.enable")
    local meta = orange_db.get_json("headers.meta")
    local selectors = orange_db.get_json("headers.selectors")
    local ordered_selectors = meta and meta.selectors

    if not enable or enable ~= true or not meta or not ordered_selectors or not selectors then
        return
    end

    for i, sid in ipairs(ordered_selectors) do
        local selector = selectors[sid]
        ngx.log(ngx.INFO, "==[Headers][START SELECTOR:", sid, "][NAME:",selector.name,']')
        if selector and selector.enable == true then
            local selector_pass
            if selector.type == 0 then -- 全流量选择器
                selector_pass = true
            else
                selector_pass = judge_util.judge_selector(selector, "headers")-- selector judge
            end

            if selector_pass then
                if selector.handle and selector.handle.log == true then
                    ngx.log(ngx.INFO, "[Headers][PASS-SELECTOR:", sid, "]")
                end

                local stop = filter_rules(sid, "headers")
                if stop then -- 不再执行此插件其他逻辑
                    return
                end
            else
                if selector.handle and selector.handle.log == true then
                    ngx.log(ngx.INFO, "[Headers][NOT-PASS-SELECTOR:", sid, "] ")
                end
            end

            -- if continue or break the loop
            if selector.handle and selector.handle.continue == true then
                -- continue next selector
            else
                break
            end
        end
    end
end

通过看源代码，可以发现，这里的一个个插件，优先像Java里面的过滤器或者PHP里面的中间件，利用装饰者模式，在外面不断套壳。 然后这里调用了处理逻辑函数filter_rules

local function filter_rules(sid, plugin)
    local rules = orange_db.get_json(plugin .. ".selector." .. sid .. ".rules")

    if not rules or type(rules) ~= "table" or #rules <= 0 then
        return false
    end

    for i, rule in ipairs(rules) do
        if rule.enable == true then
            -- judge阶段
            local pass = judge_util.judge_rule(rule, "headers")
            -- handle阶段
            if pass then
                -- extract阶段
                headers_util:set_headers(rule)
            end
        end
    end

    return false
end

如果满足条件，就会写入headers， 接着让我们回到Orange的主代码中看看rewrite方法是如何在上下文中被使用的。

function Orange.rewrite()
    ngx.ctx.ORANGE_REWRITE_START = now()

    for _, plugin in ipairs(loaded_plugins) do
        plugin.handler:rewrite()
    end

    local now_time = now()
    ngx.ctx.ORANGE_REWRITE_TIME = now_time - ngx.ctx.ORANGE_REWRITE_START
    ngx.ctx.ORANGE_REWRITE_ENDED_AT = now_time
end

令人吃惊的是，主代码就是循环调用每个插件的rewrite方法。这就会带来一个问题，两个插件如果作用于同一个属性，那么后执行的插件就会覆盖先执行的插件的结果。

redirect

默认实现为：

function BasePlugin:redirect()
    ngx.log(ngx.DEBUG, " executing plugin \"", self._name, "\": redirect")
end

找一个例子分析一下，以RedirectHandler为例

function RedirectHandler:redirect()
    RedirectHandler.super.redirect(self)

    local enable = orange_db.get("redirect.enable")
    local meta = orange_db.get_json("redirect.meta")
    local selectors = orange_db.get_json("redirect.selectors")
    local ordered_selectors = meta and meta.selectors
    
    if not enable or enable ~= true or not meta or not ordered_selectors or not selectors then
        return
    end
    
    local ngx_var = ngx.var
    local ngx_var_uri = ngx_var.uri
    local ngx_var_host = ngx_var.http_host
    local ngx_var_scheme = ngx_var.scheme
    local ngx_var_args = ngx_var.args

    for i, sid in ipairs(ordered_selectors) do
        ngx.log(ngx.INFO, "==[Redirect][PASS THROUGH SELECTOR:", sid, "]")
        local selector = selectors[sid]
        if selector and selector.enable == true then
            local selector_pass 
            if selector.type == 0 then -- 全流量选择器
                selector_pass = true
            else
                selector_pass = judge_util.judge_selector(selector, "redirect")-- selector judge
            end

            if selector_pass then
                if selector.handle and selector.handle.log == true then
                    ngx.log(ngx.INFO, "[Redirect][PASS-SELECTOR:", sid, "] ", ngx_var_uri)
                end

                local stop = filter_rules(sid, "redirect", ngx_var_uri, ngx_var_host, ngx_var_scheme, ngx_var_args)
                local selector_continue = selector.handle and selector.handle.continue
                if stop or not selector_continue then -- 不再执行此插件其他逻辑
                    return
                end
            else
                if selector.handle and selector.handle.log == true then
                    ngx.log(ngx.INFO, "[Redirect][NOT-PASS-SELECTOR:", sid, "] ", ngx_var_uri)
                end
            end
        end
    end
end

可以发现，代码结构与rewrite非常相似。在来看看它的处理函数

local function filter_rules(sid, plugin, ngx_var_uri)
    local rules = orange_db.get_json(plugin .. ".selector." .. sid .. ".rules")
    if not rules or type(rules) ~= "table" or #rules <= 0 then
        return false
    end

    for i, rule in ipairs(rules) do
        if rule.enable == true then
            -- judge阶段
            local pass = judge_util.judge_rule(rule, "rewrite")
            -- extract阶段
            local variables = extractor_util.extract_variables(rule.extractor)

            -- handle阶段
            if pass then
                local handle = rule.handle
                if handle and handle.uri_tmpl then
                    local to_rewrite = handle_util.build_uri(rule.extractor.type, handle.uri_tmpl, variables)
                    if to_rewrite and to_rewrite ~= ngx_var_uri then
                        if handle.log == true then
                            ngx.log(ngx.INFO, "[Rewrite] ", ngx_var_uri, " to:", to_rewrite)
                        end

                        local from, to, err = ngx_re_find(to_rewrite, "[?]{1}", "jo")
                        if not err and from and from >= 1 then
                            --local qs = ngx_re_sub(to_rewrite, "[A-Z0-9a-z-_/]*[%?]{1}", "", "jo")
                            local qs = string_sub(to_rewrite, from+1)
                            to_rewrite = string_sub(to_rewrite, 1, from-1)
                            if qs then
                                local args = ngx_decode_args(qs, 0)
                                if args then 
                                    ngx_set_uri_args(args) 
                                end
                            end
                        end
                        ngx_set_uri(to_rewrite, true)
                    end
                end

                return true
            end
        end
    end

    return false
end

由此可见，实现这几个基本方法的大方法的写法几乎都是一致的，核心逻辑是放在filter_rules中的。 ​

​

所以，接下来应该分析这些核心方法中调用的handle_util、extractor_util、judge_util究竟干了些啥。

judge

让我们把视线放到utils包中，请看下面的思维导图 这是判断部分代码的调用情况，由图可知，condition中的judge函数的实现尤为重要。 那我们直接来欣赏一下源码

function _M.judge(condition)
    local condition_type = condition and condition.type
    if not condition_type then
        return false
    end

    local operator = condition.operator
    local expected = condition.value
    if not operator or not expected then
        return false
    end

    local real

    if condition_type == "URI" then
        real = ngx.var.uri
    elseif condition_type == "Query" then
        local query = ngx.req.get_uri_args()
        real = query[condition.name]
    elseif condition_type == "Header" then
        local headers = ngx.req.get_headers()
        real = headers[condition.name]
    elseif condition_type == "Cookie" then
        local cookies = ngx.ctx.__cookies__
        if cookies then
            real = cookies:get(condition.name)
        end
    elseif condition_type == "IP" then
        real =  ngx.var.remote_addr
    elseif condition_type == "Random" then
        real = ngx.now() * 1000 % 100
    elseif condition_type == "UserAgent" then
        real =  ngx.var.http_user_agent
    elseif condition_type == "Method" then
        local method = ngx.req.get_method()
        method = string_lower(method)
        if not expected or type(expected) ~= "string" then
            expected = ""
        end
        expected = string_lower(expected)
        real = method
    elseif condition_type == "PostParams" then
        local headers = ngx.req.get_headers()
        local header = headers['Content-Type']
        if header then
            local is_multipart = string_find(header, "multipart")
            if is_multipart and is_multipart > 0 then
                return false
            end
        end

        ngx.req.read_body()
        local post_params, err = ngx.req.get_post_args()
        if not post_params or err then
            ngx.log(ngx.ERR, "[Condition Judge]failed to get post args: ", err)
            return false
        end

        real = post_params[condition.name]
    elseif condition_type == "Referer" then
        real =  ngx.var.http_referer
    elseif condition_type == "Host" then
        real =  ngx.var.host
    end

    return assert_condition(real, operator, expected)
end

这一大段根据输入进来的判断条件的类型来赋予待比较部分实际的值，根据代码可知，主要有

• URI
  • 获取值的方法
    • real = ngx.var.uri
• Query
  • 获取值的方法为
    • local query = ngx.req.get_uri_args()
    • real = query[condition.name]
• Header
  • 取值
    • local headers = ngx.req.get_headers()
    • real = headers[condition.name]
• Cookie
  • 取值
    • local cookies = ngx.ctx.cookies
    • real = cookies:get(condition.name)
• IP
  • 取值
    • real = ngx.var.remote_addr
• Random
  • 取值
    • real = ngx.now() * 1000 % 100
• UserAgent
  • 取值
    • real = ngx.var.http_user_agent
• Method
  • 取值
    • local method = ngx.req.get_method()
    • real = method
• PostParams
  • 取值
    • ngx.req.read_body()
    • local post_params, err = ngx.req.get_post_args()
    • real = post_params[condition.name]
• Referer
  • 取值
    • real = ngx.var.http_referer
• Host
  • 取值
    • real = ngx.var.host

遗憾的是，这仅仅只是数据准备部分，真正核心的是assert_condition，还得再看看该函数的实现

local function assert_condition(real, operator, expected)
    if not real then
        ngx.log(ngx.ERR, string_format("assert_condition error: %s %s %s", real, operator, expected))
        return false
    end

    if operator == 'match' then
        if ngx_re_find(real, expected, 'isjo') ~= nil then
            return true
        end
    elseif operator == 'not_match' then
        if ngx_re_find(real, expected, 'isjo') == nil then
            return true
        end
    elseif operator == "=" then
        if real == expected then
            return true
        end
    elseif operator == "!=" then
        if real ~= expected then
            return true
        end
    elseif operator == '>' then
        if real ~= nil and expected ~= nil then
            expected = tonumber(expected)
            real = tonumber(real)
            if real and expected and real > expected then
                return true
            end
        end
    elseif operator == '>=' then
        if real ~= nil and expected ~= nil then
            expected = tonumber(expected)
            real = tonumber(real)
            if real and expected and real >= expected then
                return true
            end
        end
    elseif operator == '<' then
        if real ~= nil and expected ~= nil then
            expected = tonumber(expected)
            real = tonumber(real)
            if real and expected and real < expected then
                return true
            end
        end
    elseif operator == '<=' then
        if real ~= nil and expected ~= nil then
            expected = tonumber(expected)
            real = tonumber(real)
            if real and expected and real <= expected then
                return true
            end
        end
    elseif operator == '%' then
        local mod_num = 0;
        local value = {}
        local idx = 1
        --expected like: 50|1,2,3
        for i in re_gmatch(expected, "(\\d+)", "jsio") do
            if idx == 1 then
                mod_num = tonumber(i[1])
            else
                value[i[1]] = true
            end
            idx = idx + 1
        end
        local mod_value = math.fmod(tonumber(real), mod_num)
        return value[tostring(mod_value)] == true;
    end

    return false
end

看起来是一个统一处理两个值关系的方法，包括匹配，大于，小于，大于等于，小于等于和%的比较。 ​

看完了这些，思路就很清晰了，即首先从数据库中加载rule，如果judge为真，表明符合某个特征，然后就根据该rule做相应的处理。 ​

​