征服国产生态!Ambari 完美适配银河麒麟 V20 实战指南:打通大数据管理“最后一公里”
背景挑战
随着国产化替代进程加速,银河麒麟(Kylin)V20 操作系统已成为关键领域的主流选择。然而,将 Apache Ambari 这一成熟的大数据集群管理平台迁移至 Kylin V20,面临 依赖库兼容性、系统权限约束、服务启动冲突 等多重壁垒,传统安装方式几乎寸步难行。
本文价值
本指南首次系统化破解 Ambari 在 Kylin V20 上的适配困局,提供一套经生产验证的完整解决方案。
方法一统一主机的 OS 类型标识(伪装成以支持版本kylin10)
若无法更换仓库,可修改主机 OS 信息,使其与当前仓库支持的 OS 类型(如 Kylin10)一致。
-
修改主机的 OS 识别文件
在每台注册的主机上操作(如 hadoop-01):# 编辑 /etc/os-release sudo vi /etc/os-release修改
VERSION_ID和PRETTY_NAME为 Kylin10:NAME="Kylin" VERSION="10 (SP2)" ID=kylin VERSION_ID="10" # 关键修改点 PRETTY_NAME="Kylin 10 SP2" -
同步其他 OS 标识文件
# 修改 redhat-release 文件(如有) sudo echo "Kylin release 10 (SP2)" > /etc/redhat-release -
重启 Ambari Agent 并刷新注册信息
ambari-agent restart # 在 Ambari Server 上清除旧注册缓存(谨慎操作) ambari-server refresh-host-registration --hostname hadoop-01
方法二适配kylin20
定位Ambari服务端的OS识别文件
/usr/lib/ambari-server/lib/ambari_commons/resources/os_family.json
#修改
"kylin": {
"extends" : "redhat",
"distro": [
"kylin"
],
"versions": [
10,
20
]
},
/usr/lib/ambari-server/lib/ambari_commons/os_check.py
#修改我是uos识别成kylin,正常不需要修改
#_IS_KYLIN_LINUX = os.path.exists('/etc/kylin-release')
_IS_KYLIN_LINUX = os.path.exists('/etc/os-release')
不该会报错
/usr/lib/ambari-server/lib/ambari_server/os_check_type.py
#修改
if current_os == cluster_os or current_os == 'kylin10' or cluster_os == 'kylin10' or current_os == 'redhat8' or cluster_os == 'redhat8' or current_os == 'anolis20' or cluster_os == 'anolis20' or current_os == 'bclinux21' or cluster_os == 'bclinux21' or current_os == 'kylin20' or cluster_os == 'kylin20':
sys.exit(0)
验证仓库与操作系统的映射
确保 Ambari 仓库配置中定义了 kylin20 的软件源。
-
检查仓库配置文件
在 Ambari 服务端查看仓库配置(如HDP-3.1.repo):vi /var/lib/ambari-server/resources/stacks/HDP/3.1/repos/repoinfo.xml
/var/lib/ambari-server/resources/stacks/<HDP>/<version>/repos/repoinfo.xml
<repo>
<os family="kylin20"/> <!-- 匹配 kylin20 -->
<baseurl>http://your-repo.example.com/kylin20/HDP/</baseurl>
</repo>
修改yum_manager
/usr/lib/ambari-server/lib/ambari_commons/repo_manager/yum_manager.py
#大约在112行左右
current_os = OSCheck.get_os_family() + OSCheck.get_os_major_version()
if current_os == 'kylin10' or current_os == 'kylin20' or current_os == 'centos8' or current_os == 'anolis20' or current_os == 'bclinux21':
return [package for package in available_packages + installed_packages]
else:
return [package[0] for package in available_packages + installed_packages]
......
#大约在128行左右
if current_os == 'kylin10' or current_os == 'kylin20' or current_os == 'centos8' or current_os == 'anolis20' or current_os == 'bclinux21':
cmd = "/usr/bin/yum list available --showduplicates"
if repo_filter:
cmd="/usr/bin/yum list available --showduplicates --disablerepo=* --enablerepo={0}".format(repo_filter)
p = subprocess.Popen(cmd, shell=True, stdout=subprocess.PIPE)
re = p.stdout.readlines()
for i in range(2,len(re)):
res = re[i].decode('utf-8').strip('\r\n').split()
if pkg_names and not pkg_names in res[0]:
continue
packages.append(res[0].strip().partition(".")[0])
......
#大约在162行左右
packages = []
if current_os == 'kylin10' or current_os == 'kylin20' or current_os == 'centos8' or current_os == 'anolis20' or current_os == 'bclinux21':
cmd = "/usr/bin/yum list installed --showduplicates"
p = subprocess.Popen(cmd, shell=True, stdout=subprocess.PIPE)
re = p.stdout.readlines()
for i in range(1,len(re)):
res = re[i].decode('utf-8').strip('\r\n').split()
if pkg_names and not pkg_names in res[0]:
continue
if repo_filter and not repo_filter.lower() in res[2].lower():
continue
packages.append(res[0].strip().partition(".")[0])
else:
......
# 309
current_os = OSCheck.get_os_family() + OSCheck.get_os_major_version()
if os.geteuid() == 0:
if current_os == 'kylin10' or current_os == 'kylin20' or current_os == 'centos8' or current_os == 'anolis20' or current_os == 'bclinux21':
return self.yum_installed_check_package(name)
elif os.path.exists('/etc/openEuler-release') or os.path.exists('/etc/UnionTech-release'):
return self.rpm_check_package_available(name)
else:
return self.yum_check_package_available(name)
else:
if current_os == 'kylin10' or current_os == 'kylin20' or current_os == 'centos8' or current_os == 'anolis20' or current_os == 'bclinux21':
return self.yum_installed_check_package(name)
else:
return self.rpm_check_package_available(name)
......
修改ambari-server-2.7.3.0-139.jar
/usr/lib/ambari-server/ambari-server-2.7.3.0-139.jar
version_definition.xsd
<xs:simpleType name="family-type">
<xs:restriction base="xs:string">
<xs:enumeration value="redhat6" />
<xs:enumeration value="redhat7" />
<xs:enumeration value="redhat8" />
<xs:enumeration value="kylin10" />
<!-- 添加kylin20 -->
<xs:enumeration value="kylin20" />
<xs:enumeration value="anolis20" />
<xs:enumeration value="bclinux21" />
<xs:enumeration value="redhat-ppc6" />
<xs:enumeration value="redhat-ppc7" />
<xs:enumeration value="debian6" />
<xs:enumeration value="debian7" />
<xs:enumeration value="debian9" />
<xs:enumeration value="ubuntu12" />
<xs:enumeration value="ubuntu14" />
<xs:enumeration value="ubuntu16" />
<xs:enumeration value="ubuntu18" />
<xs:enumeration value="suse11" />
<xs:enumeration value="suse12" />
<xs:enumeration value="amazonlinux2" />
</xs:restriction>
</xs:simpleType>
修改metainfo
在 Stack 的根目录中,编辑 metainfo.xml 文件以定义 Kylin20 的支持。
-
编辑文件:
sudo vi /var/lib/ambari-server/resources/stacks/HDP/3.1/metainfo.xml -
添加或修改
<os>标签:
在<os>部分添加 Kylin20 的类型和版本匹配规则:<os> <type>kylin</type> <!-- 支持 Kylin10 和 Kylin20 --> <family>kylin</family> <versions> <version>10</version> <version>20</version> <!-- 新增 Kylin20 --> </versions> <!-- 定义 OS 识别正则 --> <packageVersion>^kylin(10|20).*$</packageVersion> </os>
修改服务定义的 Metadata(可选)
如果某些服务(如 HDFS、YARN)需要在 Kylin20 上特殊配置,需在其服务的 metainfo.xml 中添加 OS 支持。
-
示例:修改 HDFS 服务的 Metadata:
sudo vi /var/lib/ambari-server/resources/stacks/HDP/3.1/services/HDFS/metainfo.xml -
添加 OS 类型限制(如有必要):
<osSpecifics> <osSpecific> <osFamily>kylin</osFamily> <osVersions> <osVersion>10</osVersion> <osVersion>20</osVersion> <!-- 允许在 Kylin20 上安装 --> </osVersions> </osSpecific> </osSpecifics>
zookeeper
/var/lib/ambari-server/resources/stacks/<HDP>/<low-version>/services/ZOOKEEPER/
<osSpecifics>
<osSpecific>
<osFamily>amazonlinux2,redhat6,redhat7,redhat8,kylin10,kylin20,anolis20,bclinux21,suse11,suse12</osFamily>
<packages>
<package>
<name>zookeeper_${stack_version}</name>
</package>
<package>
<name>zookeeper_${stack_version}-server</name>
</package>
</packages>
</osSpecific>
<osSpecific>
<osFamily>debian7,debian9,ubuntu12,ubuntu14,ubuntu16,ubuntu18</osFamily>
<packages>
<package>
<name>zookeeper-${stack_version}</name>
</package>
<package>
<name>zookeeper-${stack_version}-server</name>
</package>
</packages>
</osSpecific>
</osSpecifics>