Cloudera Manager 大数据集群搭建离线安装详细过程修改SELINUX为permissive或disable

软件环境

本教程搭建5个CDH节点，分别为

cdh1 192.168.10.101
cdh2 192.168.10.102
cdh3 192.168.10.103
cdh4 192.168.10.104
cdh5 192.168.10.105

搭建工具：ESXI6.5 vsphere client
虚拟机ISO: centos7 mini
JDK: jdk1.8.0_131
CDH: 5.12.1
CDH Parcles: archive.cloudera.com/cdh5/parcel…
CDH Manager: archive.cloudera.com/cm5/redhat/…
KUDU 1.4.0 :archive.cloudera.com/kudu/parcel…

步骤

安装虚拟机

用vsphere client新建cdh1一台虚拟机。注意：虚拟机安装选择上海时区，并修改时间。修改配置：操作系统ip、防火墙、seliux、hosts、内核参数。

IP配置

BOOTPROTO=static
IPADDR=192.168.10.101
NETMASK=255.255.255.0
GATEWAY=192.168.10.1
DNS1=8.8.8.8
DNS2=114.114.114.114
ONBOOT=yes

关闭防火墙和禁止开机自启

systemctl stop firewalld & systemctl disable firewalld

seliux配置(重启生效)

getenforce Enforcing
Enforcing

修改SELINUX为permissive或disabled。此配置重启才生效。

vi /etc/selinux/config
SELINUX=disabled

修改本地域名

如果采用局域网内域名服务器，只需要在第一步将DNS服务器配置为内网域名服务器即可。此步骤可以忽略。

vi /etc/hosts
192.168.10.101 cdh1
192.168.10.102 cdh2
192.168.10.103 cdh3
192.168.10.104 cdh4
192.168.10.105 cdh5

文件打开数(重启生效)

vi /etc/security/limits.conf

* soft nofile 65536 
* hard nofile 65536 
* soft nproc 131072 
* hard nproc 131072

修改系统总体文件打开数

echo  6553560 > /proc/sys/fs/file-max

修改swappiness (重启生效)

echo 'vm.swappiness=10'>> /etc/sysctl.conf

关闭大叶内存(重启生效)

cat << EOF >> /etc/rc.d/rc.local 
echo never > /sys/kernel/mm/transparent_hugepage/defrag 
echo never > /sys/kernel/mm/transparent_hugepage/enabled 
EOF

重启

以上所有配置完毕，关机并验证以上配置设置无误。

复制虚拟机

由于以上配置对所有虚拟机都适用，用vsphere client 复制4个虚拟机。到处OVF文件前关机

导出 OVF文件

再部署OVF文件，依次修改各虚拟机IP。

修改主机名(每个节点)

vi /etc/sysconfig/network
cdhx

免密登录

（所有节点执行）

ssh-keygen   #（所有节点执行）
ssh-copy-id -i ~/.ssh/id_rsa.pub root@cdh1 #（其他非cdh1节点执行）
cat ~/.ssh/id_rsa.pub >> ~/.ssh/authorized_keys #（cdh1执行)
scp ~/.ssh/authorized_keys root@cdhx:/root/.ssh/ #(cdh1执行)

集群脚本(cdh1执行)

集群分发命令

yum install -y rsync #(所有节点执行)

vi /usr/local/bin/xsync
#!/bin/bash
# $#：表示传递给脚本或函数的参数个数。
#1 获取输入参数个数，如果没有参数，直接退出
pcount=$#
if((pcount==0)); then
echo no args;
exit;
fi
 
#2 获取文件名称
p1=$1
fname=`basename $p1`
echo fname=$fname
 
#3 获取上级目录到绝对路径
pdir=`cd -P $(dirname $p1); pwd`
echo pdir=$pdir
 
#4 获取当前用户名称
user=`whoami`
 

#5 循环
for host in {cdh2,cdh3,cdh4,cdh5}; do
        #echo $pdir/$fname $user@$host:$pdir
        echo --------------- $host ----------------
        rsync -rvl $pdir/$fname $user@$host:$pdir
done

集群调用命令

vi /usr/local/bin/xcall

##!/bin/sh
pcount=$#
if((pcount==0));then
        echo no args...;
        exit;
fi
echo ==================cdh1==================
$@
for host in {cdh2,cdh3,cdh4,cdh5}; do
        echo ==================$host==================
        ssh $host $@
done

赋权

chmod 777 xsync.sh
chmod 777 xcall.sh

时钟同步

查看时区

[root@cdh1 ~]# xcall 'timedatectl '
==================cdh1==================
      Local time: Fri 2019-10-11 17:24:57 CST
  Universal time: Fri 2019-10-11 09:24:57 UTC
        RTC time: Fri 2019-10-11 09:24:57
       Time zone: Asia/Shanghai (CST, +0800)
     NTP enabled: no
NTP synchronized: no
 RTC in local TZ: no
      DST active: n/a

如果时区不对，先调整时区

ln -sf /usr/share/zoneinfo/Asia/Shanghai /etc/localtime

安装ntp (所有节点)

xcall 'yum install -y ntp'

修改ntp配置文件(cdh1)

vim /etc/ntp.conf
# 允许内网其他机器同步时间
restrict 192.168.10.0 mask 255.255.255.0 nomodify notrap
server 0.cn.pool.ntp.org iburst
server 1.cn.pool.ntp.org iburst
server 2.cn.pool.ntp.org iburst
server 3.cn.pool.ntp.org iburst

修改ntp配置文件(cdh2-5)

vim /etc/ntp.conf
# 允许内网其他机器同步时间
server cdh1

重启ntpd

xcall systemctl start ntpd
xcall systemctl enable ntpd

本地yum库

安装httpd,并启动

yum install -y httpd
systemctl start httpd
systemctl enable httpd

创建/var/www/html/pub/cm/5.12.1/RPMS/x86_64/目录，将以下文件上传

mkdir -p /var/www/html/pub/cm/5.12.1/RPMS/x86_64/

cloudera-manager-agent-5.12.1-1.cm5121.p0.6.el7.x86_64.rpm
cloudera-manager-daemons-5.12.1-1.cm5121.p0.6.el7.x86_64.rpm
cloudera-manager-server-5.12.1-1.cm5121.p0.6.el7.x86_64.rpm
cloudera-manager-server-db-2-5.12.1-1.cm5121.p0.6.el7.x86_64.rpm
enterprise-debuginfo-5.12.1-1.cm5121.p0.6.el7.x86_64.rpm

创建/var/www/html/pub/parcels/5.12.1/目录，将以下文件上传

mkdir -p /var/www/html/pub/parcels/5.12.1/

CDH-5.12.1-1.cdh5.12.1.p0.3-el7.parcel
CDH-5.12.1-1.cdh5.12.1.p0.3-el7.parcel.sha1
manifest.json

创建/var/www/html/pub/kudu/1.4.0目录，将以下文件上传

mkdir -p /var/www/html/pub/kudu/1.4.0

KUDU-1.4.0-1.cdh5.12.1.p0.10-el7.parcel  
KUDU-1.4.0-1.cdh5.12.1.p0.10-el7.parcel.sha1  
manifest.json

制作元数据

yum -y install createrepo
createrepo  /var/www/html/pub/

会生成/var/www/html/pub/repodata/repomd.xml文件

创建/etc/yum.repos.d/cm.repo

[cm] 
name=cm5.12 
baseurl=http://cdh1/pub/
enabled=1 
gpgcheck=0 
priority=1

yum 更新(所有节点)

xcall yum clean all
xcall yum makecache

搜索 cloudera 确定本地源配置无误

yum search cloudera

分发cm.repo文件

xsync /etc/yum.repos.d/cm.repo

jdk安装

xsync /opt/jdk-8u131-linux-x64.tar.gz
xcall 'tar zxf /opt/jdk-8u131-linux-x64.tar.gz -C /opt/'
xcall  'mv /opt/jdk1.8.0_131 /opt/jdk'

编辑环境变量并分发

vim /etc/profile
export JAVA_HOME=/opt/jdk
export CLASSPATH=.:$JAVA_HOME/jre/lib/rt.jar:$JAVA_HOME/lib/dt.jar:$JAVA_HOME/lib/tools.jar
export PATH=$PATH:$JAVA_HOME/bin

xsync /etc/profile
source /etc/profile #(各节点执行)
xcall java -version #(各节点执行)

#链接jdk 该CDH版本即使系统配置了JAVA_HOME也找不到
xcall 'mkdir /usr/lib/jvm/'
xcall 'ln -s /opt/jdk/ /usr/lib/jvm/java-8-openjdk'

mariadb 安装

yum install -y mariadb-server
#安装完成MariaDB首先启动MariaDB
systemctl start mariadb
#设置开机启动
systemctl enable mariadb

update user set host='%' where user='root';
update user set password=password('123456') where user='root'; 
flush privileges;

创建数据库

create database  cmf default character set utf8;
grant all privileges on cmf.* to 'cmf'@'%' identified by 'Cmf_123!';

create database  metastore default character set latin1;
grant all privileges on metastore.* to 'hive'@'%' identified by 'Hive_123!';

create database  amon default character set utf8;
grant all privileges on amon.* to 'amon'@'%' identified by 'Amon_123!';


create database  rman default character set utf8;
grant all privileges on rman.* to 'rman'@'%' identified by 'Rman_123!';

create database  nas default character set utf8;
grant all privileges on nas.* to 'nas'@'%' identified by 'Nas_123!';

create database  nms default character set utf8;
grant all privileges on nms.* to 'nms'@'%' identified by 'Nms_123!';

create database  oozie default character set utf8;
grant all privileges on oozie.* to 'oozie'@'%' identified by 'Oozie_123!';

create database  hue default character set utf8;
grant all privileges on hue.* to 'hue'@'%' identified by 'Hue_123!';

flush privileges;

mysql 驱动安装

xcall 'mkdir -p /usr/share/java'
cd /usr/share/java
cp /opt/mysql-connector-java-5.1.48-bin.jar /usr/share/java
mv /usr/share/java/mysql-connector-java-5.1.48-bin.jar /usr/share/java/mysql-connector-java.jar 
xsync  /usr/share/java/mysql-connector-java.jar
xcall 'chmod a+x /usr/share/java/mysql-connector-java.jar'

CDM部署

安装

yum install -y cloudera-manager-server

修改db.properties

vim /etc/cloudera-scm-server/db.properties
# The database type
# Currently 'mysql', 'postgresql' and 'oracle' are valid databases.
com.cloudera.cmf.db.type=mysql

# The database host
# If a non standard port is needed, use 'hostname:port'
com.cloudera.cmf.db.host=localhost

# The database name
com.cloudera.cmf.db.name=cmf

# The database user
com.cloudera.cmf.db.user=cmf

# The database user's password
com.cloudera.cmf.db.password=Cmf_123!

# The db setup type
# By default, it is set to INIT
# If scm-server uses Embedded DB then it is set to EMBEDDED
# If scm-server uses External DB then it is set to EXTERNAL
com.cloudera.cmf.db.setupType=EXTERNAL

启动

/etc/init.d/cloudera-scm-server restart 
/etc/init.d/cloudera-scm-server status

打开网址 http://cdh1:7180 账户名密码都是admin

部署cdm-agent

1、agent安装 (时间长，可以不用xcall)

xcall 'yum -y install cloudera-manager-daemons cloudera-manager-agent' 
#2、修改客户端配置文件config.ini 
xcall 'cp /etc/cloudera-scm-agent/config.ini /etc/cloudera-scm-agent/config.inibak' 
sed -i '/server_host/s/localhost/cdh1/' /etc/cloudera-scm-agent/config.ini
#3、启动客户端 
xcall '/etc/init.d/cloudera-scm-agent restart'
xcall '/etc/init.d/cloudera-scm-agent status'

填写parcel的地址

下载分配

安装服务

依次安装服务

问题

如果有目录权限问题按照以下命令修改

xcall chown  -R hdfs:hdfs /var/lib/hadoop-hdfs
xcall chown  -R httpfs.httpfs /var/lib/hadoop-httpfs
xcall chown  -R kms.kms /var/lib/hadoop-kms
xcall chown  -R mapred:mapred /var/lib/hadoop-mapreduce
xcall chown  -R yarn:yarn /var/lib/hadoop-yarn
xcall chmod -R 755 /var/lib/hadoop-*