hbase-2.4-fully-distributed-installation

Jed
277 阅读6分钟

1. 环境依赖

hbase.apache.org/book.html#b…

1.1 Java

HBase-0.94HBase-0.98HBase 1.0-1.1HBase 1.2+HBase 2.0-2.2HBase 2.3+
JDK 6兼容兼容不兼容不兼容不兼容不兼容
JDK 7兼容兼容兼容兼容不兼容不兼容
JDK 8不兼容未测试未测试兼容兼容兼容
JDK 9不兼容不兼容不兼容不兼容不兼容未测试

1.2 Hadoop

说明：

HBase-0.x 与 Hadoop 兼容性

HBase-0.92.xHBase-0.94.xHBase-0.96.xHBase-0.98.x
Hadoop-0.20.205兼容不兼容不兼容不兼容
Hadoop-0.22.x兼容不兼容不兼容不兼容
Hadoop-0.23.x不兼容兼容未测试不兼容
Hadoop-1.0.x不兼容不兼容不兼容不兼容
Hadoop-1.1.x未测试兼容兼容未测试
Hadoop-2.0.x-alpha不兼容未测试不兼容不兼容
Hadoop-2.1.0-beta不兼容未测试兼容不兼容
Hadoop-2.2.0不兼容未测试兼容兼容
Hadoop-2.3.x不兼容未测试兼容兼容
Hadoop-2.4.x不兼容未测试兼容兼容
Hadoop-2.5.x不兼容未测试兼容兼容

HBase-1.x 与 Hadoop 兼容性

HBase-1.0.xHBase-1.1.xHBase-1.2.xHBase-1.3.xHBase-1.4.xHBase-1.5.xHBase-1.6.xHBase-1.7.x
Hadoop-2.4.x兼容兼容兼容兼容不兼容未测试未测试未测试
Hadoop-2.5.x兼容兼容兼容兼容不兼容未测试未测试未测试
Hadoop-2.6.0不兼容不兼容不兼容不兼容不兼容未测试未测试未测试
Hadoop-2.6.1+未测试未测试兼容兼容不兼容未测试未测试未测试
Hadoop-2.7.0不兼容不兼容不兼容不兼容不兼容未测试未测试未测试
Hadoop-2.7.1+未测试未测试兼容兼容兼容未测试未测试未测试
Hadoop-2.7.7+未测试未测试未测试未测试未测试兼容不兼容不兼容
Hadoop-2.8.[0-4]未测试未测试未测试未测试未测试不兼容不兼容不兼容
Hadoop-2.8.5+未测试未测试未测试未测试未测试兼容兼容兼容
Hadoop-2.9.[0-1]未测试未测试未测试未测试未测试不兼容不兼容不兼容
Hadoop-2.9.2+未测试未测试未测试未测试未测试兼容兼容兼容
Hadoop-2.10.x未测试未测试未测试未测试未测试未测试兼容兼容

HBase-2.x 与 Hadoop 兼容性

HBase-2.0.xHBase-2.1.xHBase-2.2.xHBase-2.3.xHBase-2.4.xHBase-2.5.x
Hadoop-2.6.1+兼容不兼容不兼容未测试未测试未测试
Hadoop-2.7.[0-6]不兼容不兼容不兼容未测试未测试未测试
Hadoop-2.7.7+兼容兼容不兼容未测试未测试未测试
Hadoop-2.8.[0-2]不兼容不兼容不兼容未测试未测试未测试
Hadoop-2.8.[3-4]兼容兼容不兼容未测试未测试未测试
Hadoop-2.8.5+兼容兼容兼容未测试未测试未测试
Hadoop-2.9.[0-1]未测试不兼容不兼容未测试未测试未测试
Hadoop-2.9.2+未测试未测试兼容未测试未测试未测试
Hadoop-2.10.[0-1]未测试未测试未测试兼容兼容不兼容
Hadoop-2.10.2+未测试未测试未测试兼容兼容兼容
Hadoop-3.0.[0-2]不兼容不兼容不兼容未测试未测试未测试
Hadoop-3.0.3+不兼容兼容不兼容未测试未测试未测试
Hadoop-3.1.0不兼容不兼容不兼容不兼容不兼容不兼容
Hadoop-3.1.1+不兼容兼容兼容兼容兼容不兼容
Hadoop-3.2.[0-2]未测试未测试未测试兼容兼容不兼容
Hadoop-3.2.3+未测试未测试未测试兼容兼容兼容
Hadoop-3.3.[0-1]未测试未测试未测试兼容兼容不兼容
Hadoop-3.3.2+未测试未测试未测试兼容兼容兼容

2. 服务规划

2.1 已有环境

服务器

主机名操作系统CPU内存磁盘管理员用户IP
adp-01CentOS-9-Stream-arm642核2线程16GB100GBadmin10.0.0.24
adp-02CentOS-9-Stream-arm642核2线程16GB100GBadmin10.0.0.25
adp-03CentOS-9-Stream-arm642核2线程16GB100GBadmin10.0.0.26

已有服务

组件版本adp-01角色adp-02角色adp-03角色
Java8u361JavaJavaJava
Zookeeper3.5.10Zookeeper Server
Zookeeper Client		Zookeeper Server
Zookeeper Client		Zookeeper Server
Zookeeper Client
hadoop3.3.1NameNode
DataNode
NodeManager
SecondaryNameNode
DataNode
NodeManager
ResourceManager

DataNode
NodeManager
MapReduce Job History Server

2.2 服务规划

组件版本adp-01角色adp-02角色adp-03角色
HBase2.4.17Master
RegionServer		Master
RegionServer
RegionServer

下载地址：

  • 官网：https://archive.apache.org/dist/hbase/

  • 阿里云：https://mirrors.aliyun.com/apache/hbase

  • 清华大学：https://mirrors.tuna.tsinghua.edu.cn/apache/hbase

3. 安装部署

  1. 解压 hbase-2.4.17-bin.tar.gz
tar -zxf hbase-2.4.17-bin.tar.gz -C /opt/adp/module/
rm -f hbase-2.4.17-bin.tar.gz
cd /opt/adp/module/hbase-2.4.17

# 删除无用文件可节省磁盘，以及减少远程传输安装包的数据量（可选）
rm -f CHANGES.md LEGAL LICENSE.txt NOTICE.txt RELEASENOTES.md docs

# 备份配置文件（可选）
cp conf/hbase-env.sh conf/hbase-env.sh.default
cp conf/hbase-site.xml conf/hbase-site.xml.default
  1. 修改配置文件：conf/hbase-env.sh
# 修改 HBase 日志文件保存位置（可选）
export HBASE_LOG_DIR=/opt/adp/logs/hbase
# 修改 HBase 服务各进程 PID 保存位置（可选）
export HBASE_PID_DIR=/opt/adp/run
# 是否使用内置的 Zookeeper 管理集群，默认为 true，修嘎为 false（必须）
export HBASE_MANAGES_ZK=false
# 这行的注释放开（必须），否则启动 HBase 报错：java.lang.IllegalArgumentException: object is not an instance of declaring class
export HBASE_DISABLE_HADOOP_CLASSPATH_LOOKUP="true"
  1. 修改配置文件：conf/hbase-site.xml
<?xml version="1.0"?>
<?xml-stylesheet type="text/xsl" href="configuration.xsl"?>
<configuration>
  <property>
    <name>hbase.rootdir</name>
    <value>hdfs://adp-01:8020/hbase</value>
  </property>
  <property>
    <name>hbase.cluster.distributed</name>
    <value>true</value>
  </property>
  <property>
    <name>hbase.zookeeper.quorum</name>
    <value>adp-01,adp-02,adp-03</value>
  </property>
</configuration>
  1. 修改配置文件，conf/regionservers
adp-01
adp-02
adp-03
  1. 创建配置文件：conf/backup-masters
adp-02
  1. 分发安装包
  2. 设置环境变量（在所有节点操作）
export HBASE_HOME=/opt/adp/module/hbase-2.4.17
export PATH=$HBASE_HOME/bin:$PATH

# 保存文件后，刷新环境变量
source /etc/profile
  1. 创建日志目录和 PID 目录（可选，如修改了默认配置，则手动创建，在所有节点操作）
mkdir -p /opt/adp/logs/hbase /opt/adp/run
  1. HBase 相关目录权限所有者修改为集群管理员用户（在所有节点操作）
chown -R admin:admin /opt/adp/logs/hbase /opt/adp/run /opt/adp/module/hbase*
  1. 编写 HBase 集群启停止脚本 adp-hbase
#!/bin/bash

HBASE_MASTER="adp-01 adp-02"
HBASE_REGION_SERVERS="adp-01 adp-02 adp-03"

source /etc/profile

operations="start stop restart jps"

if [[ $# -ne 1 || ! $operations =~ $1 ]]; then
    echo "
Usage: adp-hbase operations

The following operations are supported:

  $operations

Your args are: $1
"
  exit 1
fi

line="------------------------------------------------------------"

function start_hbase() {
  echo
  for h in $HBASE_MASTER; do
    echo "Start HMaster in $h"
    echo $line
    ssh $h "source /etc/profile;hbase-daemon.sh start master"
    echo
  done
  for h in $HBASE_REGION_SERVERS; do
    echo "Start HRegionServer in $h"
    echo $line
    ssh $h "source /etc/profile;hbase-daemon.sh start regionserver"
    echo
  done
}

function stop_hbase() {
  echo
  for h in $HBASE_REGION_SERVERS; do
    echo "Stop HRegionServer in $h"
    echo $line
    ssh $h "source /etc/profile;hbase-daemon.sh stop regionserver"
    echo
  done
  for h in $HBASE_MASTER; do
    echo "Stop HMaster in $h"
    echo $line
    ssh $h "source /etc/profile;hbase-daemon.sh stop master"
    echo
  done

}

function restart_hbase() {
  stop_hbase
  sleep 5
  start_hbase
}

function jps_hbase() {
  for h in $HBASE_REGION_SERVERS; do
    echo "HBase Java process in $h"
    echo $line
    ssh $h "jps | grep -E -w 'HMaster|HRegionServer'"
    echo
  done
}

case $1 in
"start")
  start_hbase
;;
"stop")
  stop_hbase
;;
"restart")
  restart_hbase
;;
"jps")
  jps_hbase
;;
*)
;;
esac
  1. 启动 HBase 集群

一定要确保 Hadoop 集群和 Zookeeper 集群正常运行，然后再启动 HBase 集群。

~ adp-hbase start

Start HMaster in adp-01
------------------------------------------------------------
running master, logging to /opt/adp/logs/hbase/hbase-admin-master-adp-01.out

Start HMaster in adp-02
------------------------------------------------------------
running master, logging to /opt/adp/logs/hbase/hbase-admin-master-adp-02.out

Start HRegionServer in adp-01
------------------------------------------------------------
running regionserver, logging to /opt/adp/logs/hbase/hbase-admin-regionserver-adp-01.out

Start HRegionServer in adp-02
------------------------------------------------------------
running regionserver, logging to /opt/adp/logs/hbase/hbase-admin-regionserver-adp-02.out

Start HRegionServer in adp-03
------------------------------------------------------------
running regionserver, logging to /opt/adp/logs/hbase/hbase-admin-regionserver-adp-03.out

~ adp-hbase jps
HBase Java process in adp-01
------------------------------------------------------------
6728 HMaster
7049 HRegionServer

HBase Java process in adp-02
------------------------------------------------------------
3776 HMaster
4049 HRegionServer

HBase Java process in adp-03
------------------------------------------------------------
3232 HRegionServer
  1. 测试 HBase 集群
~ hbase-shell

hbase:001:0> list
TABLE                                                                                               
0 row(s)
Took 0.2419 seconds                                                                                 
=> []

hbase:002:0> create 'person', 'name', 'age'
Created table person
Took 0.7107 seconds                                                                                 
=> Hbase::Table - person

hbase:003:0> put 'person', '001', 'name:first_name', 'tom'
Took 0.1523 seconds  

hbase:004:0> get 'person', '001'
COLUMN                     CELL                                                                     
 name:first_name           timestamp=2023-04-20T05:57:55.641, value=tom                             
1 row(s)
Took 0.0655 seconds

hbase:005:0> disable 'person'
Took 0.6974 seconds  

hbase:006:0> drop 'person'
Took 0.3865 seconds

hbase:007:0> quit
  1. 访问 WEB-UI

hbase-web-ui-master.png

hbase-web-ui-backup-master.png

安装部署成功！

avatar
文章
阅读
粉丝