一、系统准备 CentOS 7.9

[root@bigdata hadoop]# cat /etc/centos-release
CentOS Linux release 7.9.2009 (Core)

二、Java环境准备 jdk 1.8.0_181

1. 在/etc/profile文件中添加JAVA_HOME

[root@bigdata hadoop]# vim /etc/profile

export JAVA_HOME=/usr/java/jdk1.8.0_181/
export PATH=$JAVA_HOME/bin:$PATH

2. 配置文件生效

[root@bigdata hadoop]# /source /etc/profile

3. 检查配置是否生效

[root@bigdata hadoop]# java -version
java version "1.8.0_181"
Java(TM) SE Runtime Environment (build 1.8.0_181-b13)
Java HotSpot(TM) 64-Bit Server VM (build 25.181-b13, mixed mode)

三、Maven环境准备 Maven 3.6.3

1. 在/etc/profile文件中添加MAVEN_HOME

[hadoop@bigdata ~]$ more .bashrc
export MAVEN_HOME=/home/hadoop/app/maven
export PATH=$MAVEN_HOME/bin:$PATH

2. 修改中央仓库地址，加速jar包下载。

[hadoop@bigdata ~]$ more $MAVEN_HOME/conf/settings.xml

<mirror>
        <!--apache 原生版本-->
	<id>alimaven</id>
	<name>aliyun maven</name>
        <url>http://maven.aliyun.com/nexus/content/groups/public/</url>    
        <mirrorOf>central</mirrorOf> 
</mirror>
	<!--如果你是使用CDH版本进行开发的，一定要加上-->
<mirror>
        <id>cloudera</id>
	<name>cloudera maven</name>
	<url>https://repository.cloudera.com/artifactory/cloudera-repos/</url>
	<mirrorOf>central</mirrorOf>
</mirror>

3. 配置文件生效

[hadoop@bigdata ~]$ /source .bashrc

4. 检查配置是否生效

[root@bigdata hadoop]# mvn -v
Apache Maven 3.6.3 (cecedd343002696d0abb50b32b541b8a6ba2883f)
Maven home: /home/hadoop/app/maven
Java version: 1.8.0_181, vendor: Oracle Corporation, runtime: /usr/java/jdk1.8.0_181/jre
Default locale: zh_CN, platform encoding: UTF-8
OS name: "linux", version: "3.10.0-1160.el7.x86_64", arch: "amd64", family: "unix"

四、安装cmake 3.22.4

1. yum卸载已安装cmake 版本低

[root@bigdata ~]# yum erase cmake

2. 编译cmake-3.22.4

[root@bigdata cmake-3.22.4]# ./configure

3. 安装cmake-3.22.4

[root@bigdata cmake-3.22.4]# make && make install

4. 验证

[root@bigdata sourcecode]# cmake -version
cmake version 3.22.4

五、安装 snappy-1.1.3

1. 编译snappy-1.1.3

[root@bigdata snappy-1.1.3]# ./configure

2. 安装snappy-1.1.3

[root@bigdata snappy-1.1.3]# make && make install

3. 验证

[root@bigdata sourcecode]# ls -lh /usr/local/lib |grep snappy
-rw-r--r-- 1 root root 511K 4月  27 23:23 libsnappy.a
-rwxr-xr-x 1 root root  955 4月  27 23:23 libsnappy.la
lrwxrwxrwx 1 root root   18 4月  27 23:23 libsnappy.so -> libsnappy.so.1.3.0
lrwxrwxrwx 1 root root   18 4月  27 23:23 libsnappy.so.1 -> libsnappy.so.1.3.0
-rwxr-xr-x 1 root root 253K 4月  27 23:23 libsnappy.so.1.3.0

六、安装 ProtocolBuffer 2.5.0

1. 编译protobuf-2.5.0

[root@bigdata protobuf-2.5.0]# ./configure

2. 安装protobuf-2.5.0

[root@bigdata protobuf-2.5.0]# make && make install

3. 验证

[root@bigdata sourcecode]# protoc --version
libprotoc 2.5.0

七、编译hadoop3.2.3

1. 执行编译

[hadoop@bigdata hadoop-3.2.3-src]$ mvn clean package -Pdist,native -DskipTests -Dtar -Dbundle.snappy -Dsnappy.lib=/usr/local/lib 

#参数说明：

- Pdist,native ：把重新编译生成的hadoop动态库； 
- DskipTests ：跳过测试 
- Dtar ：最后把文件以tar打包 
- Dbundle.snappy ：添加snappy压缩支持【默认官网下载的是不支持的】
- Dsnappy.lib=/usr/local/lib ：指snappy在编译机器上安装后的库路径

2. 编译结束，找安装包

[hadoop@bigdata target]$ pwd
/home/hadoop/sourcecode/hadoop-3.2.3-src/hadoop-dist/target
[hadoop@bigdata target]$ ll
-rw-rw-r-- 1 hadoop hadoop 420124639 4月  28 14:14 hadoop-3.2.3.tar.gz

3. 检查native Hadoop和压缩库，可见已支持Snappy压缩

[hadoop@bigdata app]$ hadoop checknative
2022-04-28 14:51:24,184 INFO bzip2.Bzip2Factory: Successfully loaded & initialized native-bzip2 library system-native
2022-04-28 14:51:24,191 INFO zlib.ZlibFactory: Successfully loaded & initialized native-zlib library
2022-04-28 14:51:24,233 WARN erasurecode.ErasureCodeNative: ISA-L support is not available in your platform... using builtin-java codec where applicable
2022-04-28 14:51:24,433 INFO nativeio.NativeIO: The native code was built without PMDK support.
Native library checking:
hadoop:  true /home/hadoop/sourcecode/hadoop-3.2.3-src/hadoop-dist/target/hadoop-3.2.3/lib/native/libhadoop.so.1.0.0
zlib:    true /lib64/libz.so.1
zstd  :  false 
snappy:  true /home/hadoop/sourcecode/hadoop-3.2.3-src/hadoop-dist/target/hadoop-3.2.3/lib/native/libsnappy.so.1
lz4:     true revision:10301
bzip2:   true /lib64/libbz2.so.1
openssl: true /lib64/libcrypto.so
ISA-L:   false libhadoop was built without ISA-L support
PMDK:    false The native code was built without PMDK support.