阅读 539

ClouderManager环境搭建【CM安装】

提要

目前安装大数据集群环境有三种方式,第一种方式是基于Apache开源社区也就是tar包,如我们之前的章节提及的HDFS和YARN等组件的安装。第二种方式和第三种方式基于第三方组件【部分功能收费】,分别是Clouder Manager和HDP的方式,由于目前Clouder Manager使用的更多,我们基于官方文档提供Clouder Manager的环境搭建。上节我们按官方文档配置好服务环境。接下来我们开始软件安装。

准备工作

软件准备

# 扩展一下,为啥不用tar包安装,由于tar包安装比较困难【编译安装】,我们选择rpm包安装。
# clouder manager
> sudo wget https://archive.cloudera.com/cm6/6.3.1/redhat7/yum/cloudera-manager.repo -P /etc/yum.repos.d/
> sudo rpm --import https://archive.cloudera.com/cm6/6.3.1/redhat7/yum//RPM-GPG-KEY-cloudera
> sudo yum update

# mysql server
> cd /opt/package
> wget http://repo.mysql.com/mysql-community-release-el7-5.noarch.rpm
> sudo rpm -ivh mysql-community-release-el7-5.noarch.rpm
> sudo yum update
复制代码

安装jdk

# Windows环境下载
链接:https://pan.baidu.com/s/1XYEqXMBpdjCv9DgA0jaMcQ 
提取码:0hny
复制代码
#安装上传软件
> sudo yum -y install lrzsz
> mkdir -p /opt/package
> cd /opt/package
> rz jdk-8u261-linux-x64.tar.gz
> tar -zxvf jdk-8u261-linux-x64.tar.gz -C /opt/software

#修改各个服务器中/OPT的权限
> sudo chown -R daiyongjun:daiyongjun /opt

> sudo scp -r ./software daiyongjun@cn.hadoop2.cluster.slave1:/opt
> sudo scp -r ./software daiyongjun@cn.hadoop2.cluster.slave2:/opt
> sudo scp -r ./software daiyongjun@cn.hadoop2.cluster.slave3:/opt
> sudo scp -r ./software daiyongjun@cn.hadoop2.cluster.slave4:/opt

# 修改各个服务器中java环境
> sudo vi /etc/profile
#java配置
export JAVA_HOME=/opt/software/jdk1.8.0_261
export CLASSPATH=$:CLASSPATH:$JAVA_HOME/lib/ 
export PATH=$PATH:$JAVA_HOME/bin

> source /etc/profile
复制代码

安装cloudera manager server

> sudo yum install cloudera-manager-daemons cloudera-manager-agent cloudera-manager-server
复制代码

安装的过程中我们发现国内网速太慢了,安装太耗时了。我们可能得考虑下载国内的RPM包。由此我们希望构建一个离线的yum源,如何构建本地yum源

构建cloudera manager server 本地yum源

# 把rpm包及其相关依赖全部都下载到本地,保存好
> cd /opt/package
> mkdir -p /opt/package/clouder-manager
> cd /opt/package/clouder-manager
# --continue 断点续传,终止后可继上次后再下载
> wget --continue  https://archive.cloudera.com/cm6/6.3.1/redhat7/yum/RPMS/x86_64/ -P /opt/package/clouder-manager

# 如果下载若干rpm包特别慢,可以找国内的RPM包【我已经下载好了】
# 链接:https://pan.baidu.com/s/1lIt8T3pqtj2pXoHCukpr3Q 
# 提取码:bh9a
> rz cloudera-manager-agent-6.3.1-1466458.el7.x86_64.rpm
> rz cloudera-manager-daemons-6.3.1-1466458.el7.x86_64.rpm
> rz cloudera-manager-server-6.3.1-1466458.el7.x86_64.rpm
> rz cloudera-manager-server-db-2-6.3.1-1466458.el7.x86_64.rpm
> rz enterprise-debuginfo-6.3.1-1466458.el7.x86_64.rpm
> rz oracle-j2sdk1.8-1.8.0+update181-1.x86_64.rpm

# 重新构建本地yum源数据
> sudo createrepo /opt/repo/
> cd /opt/repo
> ll | grep repodata
repodata【存在表示成功】

# 清除yum源加载内存信息
> sudo yum clean all
> sudo yum repolist
复制代码

三、安装cloudera manager server

通过本地yum源安装cloudera manager server

sudo yum install cloudera-manager-daemons cloudera-manager-agent cloudera-manager-server
复制代码

安装mysql server

# 安装mysql server
> sudo yum install mysql-server
> sudo systemctl start mysqld
> sudo systemctl stop mysqld
复制代码

根据安装软件创建不同用户

ServiceDatabaseUser
Cloudera Manager Serverscmscm
Activity Monitoramonamon
Reports Managerrmanrman
Huehuehue
Hive Metastore Servermetastorehive
Sentry Serversentrysentry
Cloudera Navigator Audit Servernavnav
Cloudera Navigator Metadata Servernavmsnavms
Oozieoozieoozie
> sudo mysql
# CREATE DATABASE <database> DEFAULT CHARACTER SET utf8 DEFAULT COLLATE utf8_general_ci;
# GRANT ALL ON <database>.* TO '<user>'@'%' IDENTIFIED BY '<password>';
# 配置scm
mysql> CREATE DATABASE scm DEFAULT CHARACTER SET utf8 DEFAULT COLLATE utf8_general_ci;
mysql> GRANT ALL ON scm.* TO 'scm'@'%' IDENTIFIED BY '1qaz2wsx';
mysql> flush privileges;
mysql> exit

# 如果添加了%号的用户依然无法访问mysql,可能是user表存在用户名为空的账号相互影响导致的
mysql> GRANT ALL ON scm.* TO 'scm'@'%' IDENTIFIED BY '1qaz2wsx';
mysql> DELETE FROM USER WHERE User="";
mysql> flush privileges;
复制代码

在需要安装mysql的服务器上安装mysql-connector

> cd /opt/package
> wget https://dev.mysql.com/get/Downloads/Connector-J/mysql-connector-java-5.1.46.tar.gz
> tar zxvf mysql-connector-java-5.1.46.tar.gz -C /opt/software
> sudo mkdir -p /usr/share/java/
> cd /opt/software/mysql-connector-java-5.1.46
> sudo cp mysql-connector-java-5.1.46-bin.jar /usr/share/java/mysql-connector-java.jar
复制代码

配置cloudera manager server相关数据库信息

# sudo /opt/cloudera/cm/schema/scm_prepare_database.sh [options] <databaseType> <databaseName> <databaseUser> <password>
> sudo /opt/cloudera/cm/schema/scm_prepare_database.sh mysql scm scm 1qaz2wsx
复制代码

启动cloudera manager server

> sudo systemctl start cloudera-scm-server
> sudo tail -f /var/log/cloudera-scm-server/cloudera-scm-server.log
# 登录http://<server_host>:7180  admin/admin
复制代码

关闭cloudera manager server 的 auto-tls

cloudera manager server 默认是开启auto-tls。如何确定你的clouder manager server是否开启了auto-tls?最直观的就是在web页面访问http://<server_host>:7180,观察页面是否会跳转到https://<server_host>:7183。那我们为啥要关闭auto-tls呢?https不是很好。由于auto-tls开启会造成cloudera manager server无法接收 Agent 发出的检测信号,会导致Agent的无法成功安装。 进入主页 管理 >> 设定 >> 安全性 >> Automatic configuration of TLS for services 【选择 No automatic configuration of TLS for services】【关闭TLS】

引用相关文章

cloudera : 官方安装Documentation

文章分类
阅读
文章标签