用户的安全基线要求系统中只能使用nginx作为网关,openresty都不行,kong、apisix就更不用谈了。因此只能自己编译构建带插件的nginx镜像。
用到的模块是lua-nginx-module和headers-more-nginx-module。构建过程中主要遇到的问题是版本兼容问题。
验证时用到的 nginx.conf
worker_processes 1;
error_log /var/log/nginx/error.log warn;
pid /var/run/nginx.pid;
events {
worker_connections 1024;
}
http {
include /usr/local/nginx/conf/mime.types;
default_type application/octet-stream;
# Lua 模块配置
lua_package_path "/usr/local/share/lua/5.1/?.lua;;";
lua_package_cpath "/usr/local/lib/lua/5.1/?.so;;";
server {
listen 80;
server_name localhost;
# 测试 more-header 模块:设置自定义响应头
location /test-more-headers {
# more_set_headers 是 more-header 模块的指令
more_set_headers 'X-Test-More-Header: Hello from more-header module';
more_set_headers 'X-Server-Version: Nginx-1.24.0';
return 200 'Check response headers for more-header test\n';
}
# 测试 Lua 模块:执行 Lua 脚本
location /test-lua {
# content_by_lua_block 是 Lua 模块的指令
content_by_lua_block {
ngx.say("Hello from Lua module!")
ngx.say("Current time: ", os.date("%Y-%m-%d %H:%M:%S"))
ngx.say("Server address: ", ngx.var.server_addr)
}
}
# 测试组合功能:Lua 动态设置头 + more-header 追加头
location /test-combined {
# Lua 动态生成内容和头
content_by_lua_block {
ngx.header["X-Lua-Generated"] = "true"
ngx.say("This response uses both modules")
}
# more-header 追加额外头
more_set_headers 'X-Combined-Mode: Lua+more-header';
}
}
}
Dockerfile
# 基于Alpine Linux构建
FROM alpine:3.18 as builder
# 更换为国内镜像源
RUN sed -i 's/dl-cdn.alpinelinux.org/mirrors.aliyun.com/g' /etc/apk/repositories
# 安装编译依赖
RUN apk add --no-cache \
gcc \
g++ \
make \
libc-dev \
pcre-dev \
zlib-dev \
openssl-dev \
lua-dev \
luajit-dev \
git \
wget
# 定义版本
ENV NGINX_VERSION=1.24.0
ENV LUA_NGINX_MODULE_VERSION=0.10.25
ENV NGX_DEVEL_KIT_VERSION=0.3.3
ENV NGX_HTTP_MORE_MODULE_VERSION=0.34
ENV LUAJIT_LIB=/usr/lib
ENV LUAJIT_INC=/usr/include/luajit-2.1
ENV RESTY_CORE_VERSION=0.1.27
ENV RESTY_LRUCACHE_VERSION=0.13
# 下载并解压源码;完成之后,目录下应该有6个文件:(1)nginx-1.24.0 (2)ngx_devel_kit-0.3.3 (4)headers-more-nginx-module-0.34 (3)lua-nginx-module-0.10.25 (5)lua-resty-core (6)lua-resty-lrucache-0.13
# 1 wget http://nginx.org/download/nginx-$NGINX_VERSION.tar.gz
# 2 wget https://github.com/simpl/ngx_devel_kit/archive/v$NGX_DEVEL_KIT_VERSION.tar.gz
# 3 wget https://github.com/openresty/lua-nginx-module/archive/v$LUA_NGINX_MODULE_VERSION.tar.gz
# 4 wget --timeout=30 --tries=3 https://github.com/openresty/headers-more-nginx-module/archive/v$NGX_HTTP_MORE_MODULE_VERSION.tar.gz
# 5 wget https://github.com/openresty/lua-resty-core/archive/v$RESTY_CORE_VERSION.tar.gz
# 6 wget --timeout=30 --tries=3 https://github.com/openresty/lua-resty-lrucache/archive/v$RESTY_LRUCACHE_VERSION.tar.gz
COPY . .
# 编译Nginx
RUN cd /nginx-$NGINX_VERSION && \
./configure \
--prefix=/usr/local/nginx \
--with-http_ssl_module \
--with-http_v2_module \
--with-http_realip_module \
--with-http_addition_module \
--with-http_sub_module \
--with-http_dav_module \
--with-http_flv_module \
--with-http_mp4_module \
--with-http_gunzip_module \
--with-http_gzip_static_module \
--with-http_random_index_module \
--with-http_secure_link_module \
--with-http_stub_status_module \
--with-http_auth_request_module \
--with-mail \
--with-mail_ssl_module \
--with-stream \
--with-stream_ssl_module \
--with-stream_realip_module \
--add-module=../ngx_devel_kit-$NGX_DEVEL_KIT_VERSION \
--add-module=../lua-nginx-module-$LUA_NGINX_MODULE_VERSION \
--add-module=../headers-more-nginx-module-$NGX_HTTP_MORE_MODULE_VERSION && \
make && \
make install
# 构建最终镜像
FROM alpine:3.18
RUN sed -i 's/dl-cdn.alpinelinux.org/mirrors.aliyun.com/g' /etc/apk/repositories
# 安装运行时依赖
RUN apk add --no-cache \
pcre \
zlib \
openssl \
lua \
luajit \
ca-certificates
# 从构建阶段复制Nginx
COPY --from=builder /usr/local/nginx /usr/local/nginx
# 复制resty.core到Lua标准库路径
RUN mkdir -p /usr/local/share/lua/5.1/resty
COPY /lua-resty-core/lib/resty/ /usr/local/share/lua/5.1/resty/
COPY /lua-resty-lrucache-0.13/lib/resty/ /usr/local/share/lua/5.1/resty/
# 配置Lua模块路径(覆盖所有可能的搜索路径)
ENV LUA_PACKAGE_PATH="/usr/local/share/lua/5.1/?.lua;/usr/share/lua/5.1/?.lua;/usr/local/lib/lua/5.1/?.lua;;"
ENV PATH="/usr/local/nginx/sbin:${PATH}"
# 创建用户和目录
RUN addgroup -S nginx && adduser -D -S -h /var/cache/nginx -s /sbin/nologin -G nginx nginx && \
mkdir -p /var/log/nginx && \
chown -R nginx:nginx /var/log/nginx && \
mkdir -p /var/cache/nginx && \
chown -R nginx:nginx /var/cache/nginx
# 复制配置文件
COPY nginx.conf /usr/local/nginx/conf/nginx.conf
EXPOSE 80
CMD ["nginx", "-g", "daemon off;"]
TODO: 使用dumb-init解决nginx在容器中的优雅退出
验证过程
部署
docker run -d -p 80:80 --name ngx ngx:1.24.0
验证more-header模块
预期有自定义的响应头
curl -I http://localhost:8080/test-more-headers
HTTP/1.1 200 OK
Server: nginx/1.24.0
Date: Sat, 20 Sep 2025 06:17:28 GMT
Content-Type: application/octet-stream
Content-Length: 44
Connection: keep-alive
X-Test-More-Header: Hello from more-header module
X-Server-Version: Nginx-1.24.0
验证lua模块
预期返回lua脚本生成的动态内容
curl http://localhost:8080/test-lua
Hello from Lua module!
Current time: 2025-09-20 06:23:03
Server address: 172.17.0.2
验证组合功能
curl -i http://localhost:80/test-combined
HTTP/1.1 200 OK
Server: nginx/1.24.0
Date: Sat, 20 Sep 2025 06:25:06 GMT
Content-Type: application/octet-stream
Transfer-Encoding: chunked
Connection: keep-alive
X-Lua-Generated: true
X-Combined-Mode: Lua+more-header
This response uses both modules
在Kubernetes集群中直接使用nginx的问题
“直接使用nginx”是指将nginx二进制程序丢到镜像中运行(不使用controller)。
虽然nginx性能很好,但是在kubernetes中直接使用是有些问题的:
- 上游使用Service IP
- 不能对Pod做主动健康检测
- 无法使用IP亲和,因为无法区分后端副本
- 配置无法动态下发,至少需要手动更新nginx.conf然后reload。在集群规模增大或者需要频繁操作的场景这是不可接受的
worker_processes设置为auto时,在容器环境中通常不能正确识别cgroup 的 CPU 限制- 当 Nginx 收到
SIGTERM信号时,默认不会优雅退出,而是会立即终止所有进程(包括正在处理的连接),可能导致客户端请求被中断。SIGQUIT才是 Nginx 定义的优雅退出信号,会等待所有活跃连接处理完毕后再退出。可以用dumb-init - 必须在nginx.conf中使用Service IP;如果使用服务名,比如一个Service,删掉之后又创建了新的,nginx中解析到的IP还是之前那个。