如果你用 calico ipip 提供 kubevirt 集群，再基于虚拟机内部部署 calico ipip，那么默认是行不通的
vlxan 也不行
关键证据
-
cali接口收到IPIP包： 11:54:39.991134 IP 10.198.0.70 > 10.198.0.185: 172.26.157.201 > 172.26.29.211: ICMP echo request
-
bond0完全没收到：tcpdump输出为空，没有任何包
-
罪魁祸首在这里： -A cali-fw-cali85d18f22a70 -p ipencap -m comment --comment "Drop IPinIP encapped packets originating in workloads" -j DROP
Calico明确禁止workload（包括VM）发出IPIP包！
原因分析
这是Calico的安全机制：
- 外层Calico自己使用IPIP协议做封装
- 为了防止安全问题，禁止VM/Pod内部发送IPIP包
- 你的VM内部Kubernetes也用IPIP，冲突了！
vxlan 也不行
# iptables-save | grep cali85d18f22a70 | grep -E "4789|vxlan|VXLAN"
# Warning: iptables-legacy tables present, use iptables-legacy-save to see them
-A cali-fw-cali85d18f22a70 -p udp -m comment --comment "cali:BW_NNl3pXBpkJfv5" -m comment --comment "Drop VXLAN encapped packets originating in workloads" -m multiport --dports 4789 -j DROP
# iptables -L cali-fw-cali85d18f22a70 -n -v --line-numbers
# Warning: iptables-legacy tables present, use iptables-legacy to see them
Chain cali-fw-cali85d18f22a70 (2 references)
num pkts bytes target prot opt in out source destination
1 0 0 ACCEPT all -- * * 0.0.0.0/0 0.0.0.0/0 /* cali:k1lsvanH32zq1_kz */ ctstate RELATED,ESTABLISHED
2 19 868 DROP all -- * * 0.0.0.0/0 0.0.0.0/0 /* cali:EwBpPJEfNOfbGsZ7 */ ctstate INVALID
3 404K 41M MARK all -- * * 0.0.0.0/0 0.0.0.0/0 /* cali:teQPc3t8ZuS2Xbzc */ MARK and 0xfffcffff
4 317K 35M DROP udp -- * * 0.0.0.0/0 0.0.0.0/0 /* cali:BW_NNl3pXBpkJfv5 */ /* Drop VXLAN encapped packets originating in workloads */ multiport dports 4789
5 49416 4012K DROP 4 -- * * 0.0.0.0/0 0.0.0.0/0 /* cali:wKZ0wha27-_UgL18 */ /* Drop IPinIP encapped packets originating in workloads */
6 37455 2255K cali-po-_nHAdKpHq8Fb57BpxiH7 all -- * * 0.0.0.0/0 0.0.0.0/0 /* cali:OaI-JoRDPyZRUhIl */
7 26914 1623K RETURN all -- * * 0.0.0.0/0 0.0.0.0/0 /* cali:ly77d9CkSPIvmNtM */ /* Return if policy accepted */
8 0 0 DROP all -- * * 0.0.0.0/0 0.0.0.0/0 /* cali:zi8FFdtj37xfeW2L */ /* Drop if no policies passed packet */
9 0 0 cali-pro-_fnhY7-g5R7gXVhhcbI all -- * * 0.0.0.0/0 0.0.0.0/0 /* cali:QwvWUjtWPILoy55N */
10 0 0 RETURN all -- * * 0.0.0.0/0 0.0.0.0/0 /* cali:0AIicOXgRZS_W3Dk */ /* Return if profile accepted */
11 0 0 cali-pro-_hFH-JTMTTWG6jzvshh all -- * * 0.0.0.0/0 0.0.0.0/0 /* cali:ZncFw3NQ6Igem8Dw */
12 0 0 RETURN all -- * * 0.0.0.0/0 0.0.0.0/0 /* cali:1TEGfO0CoXG1LxWU */ /* Return if profile accepted */
13 0 0 DROP all -- * * 0.0.0.0/0 0.0.0.0/0 /* cali:YrpYX-C6i6wxmIZO */ /* Drop if no profiles matched */
