前言
大家好，我是MonSterCoder，此篇文章记录在一次线上环境中，我们利用Nginx作为服务端的负载均衡，过后我们发现无论哪个客户端往服务端发送UDP(SNMP)请求，获取到的IP地址都是来自Nginx服务器的IP地址，导致相关业务无法处理。下面就带大家复现整个过程。
环境准备
2台测试虚拟机，一台部署Nginx，另一台部署UDP(SNMP)服务端
IP以及端口信息
- Nginx - 10.211.55.31:6705
- UDP(SNMP)服务 - 10.211.55.32:6705
- 本地客户端 - 10.211.55.2
部署UDP(SNMP)服务
新建一个SpringBoot项目
引入依赖
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-web</artifactId>
</dependency>
<dependency>
<groupId>org.snmp4j</groupId>
<artifactId>snmp4j</artifactId>
<version>2.6.3</version>
</dependency>
<dependency>
<groupId>org.projectlombok</groupId>
<artifactId>lombok</artifactId>
</dependency>
代码示例
参考了网上示例
@Slf4j
@Component
public class SnmpReceiver implements CommandResponder {
private Snmp snmp;
/**
* 初始化snmp
*/
private void init() throws Exception {
ThreadPool threadPool = ThreadPool.create("SNMP_THREAD", 5);
// 初始化多线程消息转发类
MultiThreadedMessageDispatcher dispatcher = new MultiThreadedMessageDispatcher(threadPool, new MessageDispatcherImpl());
Address listenAddress = GenericAddress.parse("10.211.55.32/6705"); // 本地IP与监听端口
TransportMapping<?> transport = null;
// 创建transportMapping，对TCP与UDP协议进行处理
if (listenAddress instanceof UdpAddress) {
transport = new DefaultUdpTransportMapping((UdpAddress) listenAddress);
} else {
transport = new DefaultTcpTransportMapping((TcpAddress) listenAddress);
}
// 创建snmp
snmp = new Snmp(dispatcher, transport);
// 其中要增加三种处理模型。如果snmp初始化使用的是Snmp(TransportMapping<? extends Address> transportMapping) ,就不需要增加
snmp.getMessageDispatcher().addMessageProcessingModel(new MPv1());
snmp.getMessageDispatcher().addMessageProcessingModel(new MPv2c());
snmp.listen();
}
/**
* 启动进程
*/
public void run() {
try {
init();
snmp.addCommandResponder(this);
log.info("开始监听SNMP-Trap信息");
} catch (Exception ex) {
ex.printStackTrace();
}
}
/**
* 当监听到消息时，会自动调用该方法
*/
@Override
public void processPdu(CommandResponderEvent respEvent) {
if (respEvent == null || respEvent.getPDU() == null) {
log.warn("未监听到消息！");
} else if (respEvent.getPDU().getType() != PDU.TRAP) {
log.warn("仅支持Trap消息！");
} else {
Vector<? extends VariableBinding> recVBs = respEvent.getPDU().getVariableBindings();
log.info(String.format("Type TRAP : %s", respEvent.getPDU().getType() == PDU.TRAP));
log.info(String.format("Address : %s", respEvent.getPeerAddress()));
log.info(String.format("Community : %s", new String(respEvent.getSecurityName())));
log.info(String.format("Variable : %s", recVBs));
}
}
}
@SpringBootApplication
public class UdpDemoApplication {
public static void main(String[] args) {
SpringApplication.run(UdpDemoApplication.class, args);
}
@Autowired
private SnmpReceiver snmpReceiver;
@Bean
public ApplicationRunner startSnmpReceiver() {
return args -> snmpReceiver.run();
}
}
打包项目
mvn package -Dmaven.test.skip=true
部署项目
将打包好的UDPDemo-0.0.1-SNAPSHOT.jar 上传到虚拟机，执行java -jar UDPDemo-0.0.1-SNAPSHOT.jar启动项目
[root@centos-linux ~]# java -jar UDPDemo-0.0.1-SNAPSHOT.jar
. ____ _ __ _ _
/\\ / ___'_ __ _ _(_)_ __ __ _ \ \ \ \
( ( )\___ | '_ | '_| | '_ \/ _` | \ \ \ \
\\/ ___)| |_)| | | | | || (_| | ) ) ) )
' |____| .__|_| |_|_| |_\__, | / / / /
=========|_|==============|___/=/_/_/_/
:: Spring Boot :: (v2.7.0)
2022-06-09 15:39:12.432 INFO 7078 --- [ main] com.example.udpdemo.UdpDemoApplication : Starting UdpDemoApplication v0.0.1-SNAPSHOT using Java 1.8.0_2
42 on centos-linux.shared with PID 7078 (/root/UDPDemo-0.0.1-SNAPSHOT.jar started by root in /root)
2022-06-09 15:39:12.435 INFO 7078 --- [ main] com.example.udpdemo.UdpDemoApplication : No active profile set, falling back to 1 default profile: "def
ault"
2022-06-09 15:39:13.507 INFO 7078 --- [ main] o.s.b.w.embedded.tomcat.TomcatWebServer : Tomcat initialized with port(s): 8081 (http)
2022-06-09 15:39:13.523 INFO 7078 --- [ main] o.apache.catalina.core.StandardService : Starting service [Tomcat]
2022-06-09 15:39:13.523 INFO 7078 --- [ main] org.apache.catalina.core.StandardEngine : Starting Servlet engine: [Apache Tomcat/9.0.63]
2022-06-09 15:39:13.603 INFO 7078 --- [ main] o.a.c.c.C.[Tomcat].[localhost].[/] : Initializing Spring embedded WebApplicationContext
2022-06-09 15:39:13.603 INFO 7078 --- [ main] w.s.c.ServletWebServerApplicationContext : Root WebApplicationContext: initialization completed in 1090 m
s
2022-06-09 15:39:14.066 INFO 7078 --- [ main] o.s.b.w.embedded.tomcat.TomcatWebServer : Tomcat started on port(s): 8081 (http) with context path ''
2022-06-09 15:39:14.076 INFO 7078 --- [ main] com.example.udpdemo.UdpDemoApplication : Started UdpDemoApplication in 1.995 seconds (JVM running for 2
.332)
2022-06-09 15:39:14.114 INFO 7078 --- [ main] com.example.udpdemo.SnmpReceiver : 开 始 监 听 SNMP-Trap信 息
关闭防火墙
[root@centos-linux ~]# systemctl stop firewalld
Nginx安装
本文以1.22.0版本为例
安装依赖
yum -y install gcc gcc-c++ pcre pcre-devel gd-devel openssl openssl-devel zlib zlib-devel
下载解压
[root@Nginx ~]# cd /opt
[root@Nginx opt]# wget http://nginx.org/download/nginx-1.22.0.tar.gz
[root@Nginx opt]# tar -xf nginx-1.22.0.tar.gz -C nginx
配置
[root@Nginx opt]# cd nginx
[root@Nginx nginx]# ./configure --with-stream
编译安装
[root@Nginx nginx]# make && make install
修改配置
[root@Nginx nginx]# cd /usr/local/nginx/conf
[root@Nginx conf]# vi nginx.conf
// 加入以下配置
stream {
server {
listen 6705 udp;
proxy_pass 10.211.55.32:6704;
proxy_responses 0;
}
}
启动Nginx
[root@Nginx conf]# cd /usr/local/nginx/sbin
[root@Nginx sbin]# ./nginx -c /usr/local/nginx/conf/nginx.conf
关闭防火墙
[root@Nginx conf]# systemctl stop firewalld
本地10.211.55.2测试UDP报文
代码示例
public static void main(String[] args) throws IOException {
Address targetAddress = GenericAddress.parse("udp:10.211.55.31/6705"); // Nginx地址
TransportMapping<UdpAddress> transport = new DefaultUdpTransportMapping();
Snmp snmp = new Snmp(transport);
PDU pdu = new PDU();
VariableBinding v = new VariableBinding();
v.setOid(new OID("1.21.1.2.2.3.45.5.57567.3"));//随便写的oid
v.setVariable(new OctetString("Snmp Trap V2 Test 123"));
v.setOid(new OID("1.21.1.2.2.3.45.5.57567.4"));//随便写的oid
v.setVariable(new OctetString("你好 Snmp Trap V2 Test 123"));
pdu.add(v);
pdu.setType(PDU.TRAP);
// set target
CommunityTarget target = new CommunityTarget();
target.setCommunity(new OctetString("public"));
target.setAddress(targetAddress);
// retry times when commuication error
target.setRetries(2);
target.setTimeout(1500);
target.setVersion(SnmpConstants.version2c);
// send pdu, return response
snmp.send(pdu, target);
}
返回结果
2022-06-08 21:52:52.573 INFO 70411 --- [ SNMP_THREAD.0] com.example.udpdemo.SnmpReceiver : Type TRAP : true
2022-06-08 21:52:52.573 INFO 70411 --- [ SNMP_THREAD.0] com.example.udpdemo.SnmpReceiver : Address : 10.211.55.31/45265
2022-06-08 21:52:52.573 INFO 70411 --- [ SNMP_THREAD.0] com.example.udpdemo.SnmpReceiver : Community : public
2022-06-08 21:52:52.574 INFO 70411 --- [ SNMP_THREAD.0] com.example.udpdemo.SnmpReceiver : Variable : [1.21.1.2.2.3.45.5.57567.4 = e4:bd:a0:e5:a5:bd:
20:53:6e:6d:70:20:54:72:61:70:20:56:32:20:54:65:73:74:20:31:32:33]
我们发现Address输出的IP地址为Nginx服务器的IP地址，并不是我们客户端的IP地址，为此，我们需要更改Nginx一处配置
更改Nginx配置文件
[root@Nginx sbin]# cd /usr/local/nginx/conf
[root@Nginx conf]# vi nginx.conf
// 加入以下配置
stream {
server {
listen 6705 udp; // Nginx监听端口
proxy_pass 10.211.55.31:6704; 后端UDP代理地址
proxy_responses 0;
proxy_bind $remote_addr transparent; // 加入此配置, 后端服务器会接受到来自客户端IP的请求
}
}
重启Nginx
[root@Nginx conf]# cd /usr/local/nginx/sbin
[root@Nginx sbin]# ./nginx -s reload -c /usr/local/nginx/conf/nginx.conf
重新发起测试
返回结果
2022-06-08 21:52:52.573 INFO 70411 --- [ SNMP_THREAD.0] com.example.udpdemo.SnmpReceiver : Type TRAP : true
2022-06-08 21:52:52.573 INFO 70411 --- [ SNMP_THREAD.0] com.example.udpdemo.SnmpReceiver : Address : 10.211.55.2/60439
2022-06-08 21:52:52.573 INFO 70411 --- [ SNMP_THREAD.0] com.example.udpdemo.SnmpReceiver : Community : public
2022-06-08 21:52:52.574 INFO 70411 --- [ SNMP_THREAD.0] com.example.udpdemo.SnmpReceiver : Variable : [1.21.1.2.2.3.45.5.57567.4 = e4:bd:a0:e5:a5:bd:
20:53:6e:6d:70:20:54:72:61:70:20:56:32:20:54:65:73:74:20:31:32:33]
至此我们发现已经能够正常的获取到客户端的IP，通过Nginx转发UDP后无法获取真实IP问题也完美解决。
