【SFC】学习 -- Service Function Chain 验证

雪糕是宝贝
178 阅读8分钟

image.png

开启掘金成长之旅！这是我参与「掘金日新计划 · 2 月更文挑战」的第 15 天，点击查看活动详情

查看主机对应的端口以及MAC地址

# PS0
10.0.2.105
fa:16:3e:b4:8a:90

# PS1
10.0.2,104
fa:16:3e:4c:39:ba

# PS2
10.0.3.103
fa:16:3e:54:9a:c4

# PS3
10.0.3.102
fa:16:3e:8c:12:53

# PS4
10.0.4.114
fa:16:3e:34:49:fa

# PS5
10.0.4.109
fa:16:3e:4b:35:22

# PS6
10.0.5.106
fa:16:3e:3b:11:5a

# PS7
10.0.5.107
fa:16:3e:ef:70:98

image.png

Bug 发现一个compute01计算节点的网卡都是down的

重启OVS agent服务，端口up

systemctl restart neutron-openvswitch-agent.service

image.png

SFC-SRC

systemctl stop firewalld
systemctl stop iptables

# 由于配置了两张网卡，所以加一个明细路由，否则ping 16网段的会默认走eth0
route add 10.0.16.0/24 dev eth1
  • 此时ping 10.0.16.102，在VM1上抓包如下：

image.png

  • 确保同网段的vxlan能够ping通 image.png

  • 抓取vxlan数据包

image.png

bug

image.png

  • 修改rpc的时延：

OVS原理

OVS agent在br-int桥上添加了额外的流流表来支持SFC:

  1. 出端口根据规则将流量从连接到br-int网桥的虚机流量引入br-tun，经过隧道到达下一个compute节点。
  2. 入端口规则在br-int网桥上进行匹配，将业务链流量从br-tun引入到Ibr-int桥的SF（虚机）
  3. 内部端口规则用于将业务链流量从同一个计算节点上的一个SF引导到另一个SF。
  4. 本地交换表Table 0。生成两个新的流表来处理来自SF出口端口和计算节点之间的隧道端口的传入流量。
  5. 其中流量分类表集成在br-int中，用来做流量分类。
  6. SF转发流表会下发到br-tun中，使通过tun的流量走到对应的下一跳port chain的端口，再进入br-int到对应的虚机。
  7. 组表（Group Table）。该表用于在端口对组中多个端口对之间选择多条路径进行负载均衡。如果下一跳是包含多个端口对的端口对组，则组中存在多个桶。分组动作是将分组发送到下一跳SF实例。如果下一跳端口对在其他计算节点上，则将tunnel端口的动作输出到下一跳计算节点。如果下一跳端口对在同一计算节点上，则动作为重新提交给TUN_TABLE进行本地链接处理。

Table 0

cookie=0xa5608bafc63e789d, duration=4.929s, table=0, n_packets=0, n_bytes=0, priority=30,ip,in_port="tap7a1c016c-c0",nw_src=10.0.2.0/24,nw_dst=10.0.5.0/24 actions=group:1
 
cookie=0xa5608bafc63e789d, duration=4.573s, table=0, n_packets=0, n_bytes=0, priority=30,ip,in_port="tap9bc21911-f2",nw_src=10.0.2.0/24,nw_dst=10.0.5.0/24 actions=group:3
 
cookie=0xa5608bafc63e789d, duration=4.285s, table=0, n_packets=0, n_bytes=0, priority=30,ip,in_port="tap43002f21-0b",nw_src=10.0.5.0/24,nw_dst=10.0.2.0/24 actions=group:7003

cookie=0xa5608bafc63e789d, duration=3.920s, table=0, n_packets=0, n_bytes=0, priority=30,ip,in_port="tapc39752ff-86",nw_src=10.0.5.0/24,nw_dst=10.0.2.0/24 actions=group:7001

image.png

  • in_port="tap7a1c016c-c0",nw_src=10.0.2.0/24,nw_dst=10.0.5.0/24 actions=group:1

入流量端口 tap7a1c016c-c0 匹配源地址为10.0.2.0/24 目的地址为10.0.5.0/24的流量，行为：转发至group:1

  • 查看br-int端口
[root@compute01 ~]# ovs-ofctl show br-int
OFPT_FEATURES_REPLY (xid=0x2): dpid:0000fae4e95bc149
n_tables:254, n_buffers:0
capabilities: FLOW_STATS TABLE_STATS PORT_STATS QUEUE_STATS ARP_MATCH_IP
actions: output enqueue set_vlan_vid set_vlan_pcp strip_vlan mod_dl_src mod_dl_dst mod_nw_src mod_nw_dst mod_nw_tos mod_tp_src mod_tp_dst
 1(int-br-em): addr:ee:4f:79:3a:8d:63
     config:     0
     state:      0
     speed: 0 Mbps now, 0 Mbps max
 2(patch-tun): addr:92:a5:a2:2f:7b:fb
     config:     0
     state:      0
     speed: 0 Mbps now, 0 Mbps max
 3(tapd37e2ddb-54): addr:fe:16:3e:19:7d:5c
     config:     0
     state:      0
     current:    10MB-FD COPPER
     speed: 10 Mbps now, 0 Mbps max
 4(tap43002f21-0b): addr:fe:16:3e:ef:70:98
     config:     0
     state:      0
     current:    10MB-FD COPPER
     speed: 10 Mbps now, 0 Mbps max
 5(tap1b7d577f-7b): addr:fe:16:3e:af:9f:87
     config:     0
     state:      0
     current:    10MB-FD COPPER
     speed: 10 Mbps now, 0 Mbps max
 6(tapc39752ff-86): addr:fe:16:3e:8c:12:53
     config:     0
     state:      0
     current:    10MB-FD COPPER
     speed: 10 Mbps now, 0 Mbps max
 7(tap9bc21911-f2): addr:fe:16:3e:f1:e3:1d
     config:     0
     state:      0
     current:    10MB-FD COPPER
     speed: 10 Mbps now, 0 Mbps max
 8(tapc7e55b74-b3): addr:fe:16:3e:1b:23:a5
     config:     0
     state:      0
     current:    10MB-FD COPPER
     speed: 10 Mbps now, 0 Mbps max
 9(tap7a1c016c-c0): addr:fe:16:3e:b4:8a:90
     config:     0
     state:      0
     current:    10MB-FD COPPER
     speed: 10 Mbps now, 0 Mbps max
 LOCAL(br-int): addr:fa:e4:e9:5b:c1:49
     config:     0
     state:      0
     speed: 0 Mbps now, 0 Mbps max
OFPT_GET_CONFIG_REPLY (xid=0x4): frags=normal miss_send_len=0
  • 每一跳出口的sfc报头封装, 在table:5中实现:
 cookie=0xa5608bafc63e789d, duration=7787.038s, table=5, n_packets=0, n_bytes=0, priority=0,ip,dl_dst=fa:16:3e:4c:39:ba actions=push_mpls:0x8847,set_mpls_label(511),set_mpls_ttl(255),mod_vlan_vid:5,output:"patch-tun"
 
 
 cookie=0xa5608bafc63e789d, duration=7786.682s, table=5, n_packets=0, n_bytes=0, priority=0,ip,dl_dst=fa:16:3e:34:49:fa actions=push_mpls:0x8847,set_mpls_label(509),set_mpls_ttl(253),mod_vlan_vid:3,output:"patch-tun"
 
 
 cookie=0xa5608bafc63e789d, duration=7786.393s, table=5, n_packets=0, n_bytes=0, priority=0,ip,dl_dst=fa:16:3e:4b:35:22 actions=push_mpls:0x8847,set_mpls_label(511),set_mpls_ttl(255),mod_vlan_vid:1,output:"patch-tun"
 
 
 cookie=0xa5608bafc63e789d, duration=7786.030s, table=5, n_packets=0, n_bytes=0, priority=0,ip,dl_dst=fa:16:3e:54:9a:c4 actions=push_mpls:0x8847,set_mpls_label(509),set_mpls_ttl(253),mod_vlan_vid:4,output:"patch-tun"
  • dl_dst=fa:16:3e:4c:39:ba actions=push_mpls:0x8847,set_mpls_label(511),set_mpls_ttl(255),mod_vlan_vid:5,output:"patch-tun"

image.png

fa:16:3e:4c:39:ba p1接口的mac地址，目的地址为p1接口mac地址的，封装mpls报头，经patch-tun接口转发流量至br-tun。经过vxlan到达compute02

流量到达compute02后，发现协议是mpls的，转发至table10

cookie=0xfa034db9f73e3d5f, duration=78052.721s, table=0, n_packets=1527, n_bytes=155754, idle_age=8690, hard_age=65534, priority=20,mpls actions=resubmit(,10)

 cookie=0xfa034db9f73e3d5f, duration=8303.904s, table=10, n_packets=0, n_bytes=0, priority=1,mpls,dl_vlan=4,dl_dst=fa:16:3e:4c:39:ba,mpls_label=511 actions=strip_vlan,pop_mpls:0x0800,output:"tap8120f5fc-87"
 cookie=0xfa034db9f73e3d5f, duration=8303.655s, table=10, n_packets=0, n_bytes=0, priority=1,mpls,dl_vlan=2,dl_dst=fa:16:3e:34:49:fa,mpls_label=509 actions=strip_vlan,pop_mpls:0x0800,output:"tap7b31ace2-96"
 cookie=0xfa034db9f73e3d5f, duration=8303.261s, table=10, n_packets=0, n_bytes=0, priority=1,mpls,dl_vlan=3,dl_dst=fa:16:3e:4b:35:22,mpls_label=511 actions=strip_vlan,pop_mpls:0x0800,output:"tap17e6712e-c4"
 cookie=0xfa034db9f73e3d5f, duration=8303.004s, table=10, n_packets=0, n_bytes=0, priority=1,mpls,dl_vlan=5,dl_dst=fa:16:3e:54:9a:c4,mpls_label=509 actions=strip_vlan,pop_mpls:0x0800,output:"tap9d93d5b1-69"
 cookie=0xfa034db9f73e3d5f, duration=78096.245s, table=10, n_packets=0, n_bytes=0, priority=0 actions=drop

根据标签与目的mac地址，剥离mpls报文头部

mpls,dl_vlan=4,dl_dst=fa:16:3e:4c:39:ba,mpls_label=511，actions=strip_vlan,pop_mpls:0x0800,output:"tap8120f5fc-87"

行为：剥离vlan,报文头部，转发至tap8120f5fc-87 （P1）接口

SRC虚机配置

ip route add 10.0.5.107 dev eth1
  • 在SRC虚机ping DEST虚机，能够看到匹配到流表，并进入group:1

image.png

  • 可以看到table 5也有相应的数据 image.png

  • 抓包可以看到对应的mpls报文头部，可以看到510和511的id,对应着compute01和compute02的不同虚机所走的隧道id

image.png

image.png

  • VM1-VM3 安装OVS，用于网卡转发
tar -zxvf openvswitch-2.1.0.tar.gz
cd openvswitch-2.1.0
./configure
make
make install
export PATH=$PATH:/usr/local/share/openvswitch/scripts
ovs-ctl start
ovs-vsctl show
  • 确保OVS正确安装

image.png

  • 创建网桥，并设置网络流量从eth1入，并从eth2出
ovs-vsctl add-br br-sfc
ovs-vsctl add-port br-sfc eth1
ovs-vsctl add-port br-sfc eth2

image.png

image.png

  • 确认网桥的接口MAC地址与物理网卡的MAC地址相同

image.png

  • 写入流表，1口进入 2口出

# vm1 
ovs-ofctl del-flows br-sfc
ovs-ofctl add-flow br-sfc "in_port=1 actions=output:2"

# 最后一跳的的mac地址修改为目标主机[src]
ovs-ofctl add-flow br-sfc "in_port=2 actions=mod_dl_dst:fa:16:3e:b4:8a:90,output:1"
ovs-ofctl add-flow br-sfc "priority=0 actions=NORMAL"

#vm2
ovs-ofctl del-flows br-sfc
ovs-ofctl add-flow br-sfc "in_port=1 actions=output:2"
ovs-ofctl add-flow br-sfc "in_port=2 actions=output:1"
ovs-ofctl add-flow br-sfc "priority=0 actions=NORMAL"

# vm3
# 最后一跳的的mac地址修改为目标主机[dest]
ovs-ofctl del-flows br-sfc
ovs-ofctl add-flow br-sfc "in_port=1 actions=output:2"
ovs-ofctl add-flow br-sfc "in_port=2 actions=output:1"
ovs-ofctl add-flow br-sfc "in_port=1 actions=mod_dl_dst:fa:16:3e:ef:70:98,output:2"
ovs-ofctl add-flow br-sfc "priority=0 actions=NORMAL"

ovs-ofctl add-flow br-sfc "in_port=1 actions=mod_dl_src:fa:16:3e:54:9a:c4,output:2" ovs-ofctl add-flow br-sfc "priority=0 actions=NORMAL"

image.png

  • 三台机器做相同的配置

在SRC端pingDST端，

image.png

  • DEST端:

image.png

  • 中间VM1-VM3均能收到报文

image.png

  • 查看计算节点的OVS流表

  • table = 0

image.png

  • table = 5

image.png

  • table = 10

image.png

双向链配置测试

  • 删除原有port chain 配置
openstack sfc port chain delete SFC-Chain01
  • 确保ovs 配置已经被清理掉 image.png
openstack sfc port chain create --chain-parameters symmetric=true,correlation=nsh  --flow-classifier SFC1 --port-pair-group PPG1 --port-pair-group PPG2 --port-pair-group PPG3 SFC-Chain01
  • 确认双向配置完成

image.png

DEST主机

  • 首先配置一个回程路由
ip route add 10.0.2.105/32 dev eth1
  • SRC端能够完成ping通

image.png

NSH测试

  • 先删除之前的链，增加参数 --chain-parameters symmetric=true,correlation=nsh
openstack sfc port chain create --chain-parameters symmetric=true,correlation=mpls  --flow-classifier SFC1 --port-pair-group PPG1 --port-pair-group PPG2 --port-pair-group PPG3 SFC-Chain01
  • 显示不支持nsh

image.png

avatar
文章
阅读
粉丝