鲲鹏平台centos7.6编译netty-all-4.1.X和netty-tcnative-boringssl-static-2.0.X

因为大家都知道的原因，系统需要兼容到鲲鹏平台，通过华为的兼容性扫描工具扫描，发现netty相关静态库（so）的不支持。因此需要在鲲鹏平台上（CentOS 7.6）编译netty-all和netty-tcnative-boringssl-static。
环境如下：

硬件：华为鲲鹏云 ECS
OS：CentOS 7.6 / Kernel 4.18

升级安装 OpenJDK 8 / gcc / Maven 3 / Git 3.1 gcc相关依赖：

yum install git gcc gcc-c++ make cmake libtool autoconf automake -y

3.2 完整安装openssl（不要手动升级）

yum -y install openssl*

3.3 升级cmake到3.0.0+ (这里直接用当前最新版本：3.18.0-rc4)

yum remove cmake
cd /opt/source
wget https://github.com/Kitware/CMake/releases/download/v3.18.0-rc4/cmake-3.18.0-rc4.tar.gz
tar -xvf cmake-3.18.0-rc4.tar.gz
mv cmake-3.18.0-rc4 /opt/tools/installed/
cd /opt/tools/installed/cmake-3.18.0-rc4/
./configure
make && make install
cmake -version

3.4 关于gcc可能会出现**-fsigned-char**问题

使用命令 "command -v gcc" 查找gcc安装目录（centos 通常都在 /usr/bin/gcc）
改名gcc（如 gcc-cli）"cd /usr/bin && mv gcc gcc-cli"
创建新的gcc shell "vim gcc"，填充以下内容：

#! /bin/sh

/usr/bin/gcc-cli -fsigned-char "$@"

给与执行权限 "chmod +x gcc"
按照同样的办法处理 g++

在github中下载netty-all-4.1.X（这里用的netty-all-4.1.48.Final），时间较长，可能需要不可描述的工具或者直接下载源代码zip文件

mkdir -p /opt/source && cd /opt/source
git clone --branch netty-4.1.48.Final https://github.com/netty/netty.git

mkdir -p /opt/source && cd /opt/source
wget https://github.com/netty/netty/archive/netty-4.1.48.Final.zip
unzip netty-4.1.48.Final.zip

先maven打包安装一次（肯定会报错的就是了，一般是编译到native部分就要出错了）

cd netty-netty-4.1.48.Final
mvn -X -e install -Dmaven.test.skip=true

查看pom.xml发现

<os.detection.classifierWithLikes>fedora,suse,arch</os.detection.classifierWithLikes>
<tcnative.artifactId>netty-tcnative</tcnative.artifactId>
<tcnative.version>2.0.29.Final</tcnative.version>
<tcnative.classifier>${os.detected.classifier}</tcnative.classifier>

虽然有指向arch，但是确实不兼容，因此需要先处理netty-tcnative

在github下载netty-tcnative-2.0.29.Final，同样的时间比较长

cd /opt/source
git clone --branch netty-tcnative-parent-2.0.29.Final https://github.com/netty/netty-tcnative.git

cd /opt/source
wget https://github.com/netty/netty-tcnative/archive/netty-tcnative-parent-2.0.29.Final.zip
unzip netty-tcnative-parent-2.0.29.Final.zip

先maven打包安装一次（反正任然要报错的）

cd netty-tcnative-netty-tcnative-parent-2.0.29.Final
mvn -X -e install -Dmaven.test.skip=true

配合netty-tcnative的pom和华为官方论坛和兼容性说明发现，需要处理apr、libressl、openssl:

   <aprVersion>1.6.5</aprVersion>

   <libresslVersion>3.0.2</libresslVersion>

   <opensslMinorVersion>1.1.1</opensslMinorVersion>
   <opensslPatchVersion>d</opensslPatchVersion>
   <opensslVersion>${opensslMinorVersion}${opensslPatchVersion}</opensslVersion>

首先屏蔽pom.xml中module boringssl-static（按照华为官方的说法，若编译时不屏蔽boringssl，会报tcnative openssl版本(1.0.2l)与boringssl中需要的C语言宏定义库不完全兼容导致的。并且，由于boringssl是google自己开发并自己使用，并不希望第三方软件使用，去掉此部分风险小，因此采用屏蔽此部分的方法。）（黑人问号.png）

    <!--
      Profile for building all modules. This is enabled by default so if any profile is manually specified,
      this will be disabled automatically.
    -->
    <profile>
      <id>all</id>
      <activation>
        <property>
          <name>!moduleSelector</name>
        </property>
      </activation>
      <modules>
        <module>openssl-dynamic</module>
        <module>openssl-static</module>
        <!-- <module>boringssl-static</module> -->
        <module>libressl-static</module>
      </modules>

本地编译安装apr

cd /opt/source
wget https://archive.apache.org/dist/apr/apr-1.6.5.tar.gz
tar -xvf apr-1.6.5.tar.gz
cd apr-1.6.5
./configure
make && make install

再次安装

cd netty-tcnative-netty-tcnative-parent-2.0.29.Final
mvn -X -e install -Dmaven.test.skip=true

在安装中，脚本会下载 apr-1.6.5.tar.gz 和 libressl-3.0.2.tar.gz，如果下载失败，可以手动下载后，拷贝到对应的子项目中的target目录；需要注释对应pom.xm文件中的<get src ... > 部分。以 libressl-static 为例：

cd /opt/source
wget http://ftp.openbsd.org/pub/OpenBSD/LibreSSL/libressl-3.0.2.tar.gz
cp libressl-3.0.2.tar.gz netty-tcnative-netty-tcnative-parent-2.0.29.Final/libressl-static/target/

修改pom,注释

vim netty-tcnative-netty-tcnative-parent-2.0.29.Final/libressl-static/pom.xml

 <profile>
      <id>build-libressl-non-windows</id>
      <activation>
        <os>
          <family>!windows</family>
        </os>
      </activation>
      <build>
        <plugins>
          <plugin>
            <artifactId>maven-antrun-plugin</artifactId>
            <executions>
              <!-- Download and build LibreSSL -->
              <execution>
                <id>source-libressl-non-windows</id>
                <phase>generate-sources</phase>
                <goals>
                  <goal>run</goal>
                </goals>
                <configuration>
                  <target>
                    <!-- Add the ant tasks from ant-contrib -->
                    <taskdef resource="net/sf/antcontrib/antcontrib.properties" />

                    <if>
                      <available file="${libresslCheckoutDir}" />
                      <then>
                        <echo message="LibreSSL was already downloaded, skipping the build step." />
                      </then>
                      <else>
                        <echo message="Downloading LibreSSL" />
                        <!-- get src="https://ftp.openbsd.org/pub/OpenBSD/LibreSSL/${libresslArchive}" dest="${project.build.directory}/${libresslArchive}" verbose="on" /> -->
                        <checksum file="${project.build.directory}/${libresslArchive}" algorithm="SHA-256" property="${libresslSha256}" verifyProperty="isEqual" />
                        <exec executable="tar" failonerror="true" dir="${project.build.directory}/" resolveexecutable="true">
                          <arg value="xfv" />
                          <arg value="${libresslArchive}" />
                        </exec>
                      </else>
                    </if>
                  </target>

编译"LibreSSL - Static"的时候有个坑（老版本没碰见），脚本进行make && make install后，会把生成好的"libnetty_tcnative.so"移动到"usr/local/lib/"下

make[1]: Entering directory `/opt/source/netty-tcnative-netty-tcnative-parent-2.0.29.Final/libressl-static/target/native-build'
 /usr/bin/mkdir -p '/usr/local/lib'
 /bin/sh ./libtool   --mode=install /usr/bin/install -c   libnetty_tcnative.la '/usr/local/lib'
libtool: install: /usr/bin/install -c .libs/libnetty_tcnative-@VERSION@.so /usr/local/lib/libnetty_tcnative-@VERSION@.so
libtool: install: (cd /usr/local/lib && { ln -s -f libnetty_tcnative-@VERSION@.so libnetty_tcnative.so || { rm -f libnetty_tcnative.so && ln -s libnetty_tcnative-@VERSION@.so libnetty_tcnative.so; }; })
libtool: install: /usr/bin/install -c .libs/libnetty_tcnative.lai /usr/local/lib/libnetty_tcnative.la
libtool: finish: PATH="/opt/tools/installed/apache-maven-3.6.3/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/root/bin:/sbin" ldconfig -n /usr/local/lib

所以需要手动拷贝回来，再重新打包安装

cd /opt/source/netty-tcnative-netty-tcnative-parent-2.0.29.Final/libressl-static/target/native-build/target/lib
cp /usr/local/lib/libnetty_tcnative.so .

编译netty-all

cd /opt/source/netty-netty-4.1.48.Final
mvn -X -e install -Dmaven.test.skip=true

出现jni.h头文件找不到的情况

main:
     [exec] Current OS is Linux
     [exec] Setting environment variable: CC=gcc
     [exec] Setting environment variable: AR=ar
     [exec] Setting environment variable: LIB_DIR=/opt/source/netty-netty-4.1.48.Final/transport-native-unix-common/target/native-lib-only
     [exec] Setting environment variable: OBJ_DIR=/opt/source/netty-netty-4.1.48.Final/transport-native-unix-common/target/native-objs-only
     [exec] Setting environment variable: JNI_PLATFORM=linux
     [exec] Setting environment variable: CFLAGS=-O3 -Werror -Wno-attributes -fPIC -fno-omit-frame-pointer -Wunused-variable -fvisibility=hidden
     [exec] Setting environment variable: LDFLAGS=-Wl,--no-as-needed -lrt
     [exec] Setting environment variable: LIB_NAME=libnetty-unix-common
     [exec] Executing 'make'
     [exec] The ' characters around the executable and arguments are
     [exec] not part of the command.
Execute:Java13CommandLauncher: Executing 'make'
The ' characters around the executable and arguments are
not part of the command.
     [exec] mkdir -p /opt/source/netty-netty-4.1.48.Final/transport-native-unix-common/target/native-objs-only
     [exec] gcc -o /opt/source/netty-netty-4.1.48.Final/transport-native-unix-common/target/native-objs-only/netty_unix_limits.o -c src/main/c/netty_unix_limits.c -O3 -Werror -Wno-attributes -fPIC -fno-omit-frame-pointer -Wunused-variable -fvisibility=hidden -I/include -I/include/linux
     [exec] In file included from src/main/c/netty_unix_limits.c:20:0:
     [exec] src/main/c/netty_unix_jni.h:19:17: fatal error: jni.h: No such file or directory
     [exec]  #include <jni.h>
     [exec]                  ^
     [exec] compilation terminated.
     [exec] make: *** [/opt/source/netty-netty-4.1.48.Final/transport-native-unix-common/target/native-objs-only/netty_unix_limits.o] Error 1

找到java安装目录后，修改 /etc/profile，添加

export JAVA_HOME=/usr/lib/jvm/java-1.8.0-openjdk-1.8.0.252.b09-2.el7_8.aarch64
export CPATH=$CPATH:$JAVA_HOME/include:$JAVA_HOME/include/linux
export C_INCLUDE_PATH=$C_INCLUDE_PATH:$JAVA_HOME/include:$JAVA_HOME/include/linux
export CPLUS_INCLUDE_PATH=$CPLUS_INCLUDE_PATH:$JAVA_HOME/include:$JAVA_HOME/include/linux

source后再编译