Ambari 2.7.5 源码编译一、编译准备创建一个CentOS7虚拟机安装 JDK8 安装编译基本软件安装 M

一、编译准备

创建一个CentOS7虚拟机

安装 JDK8

tar -zxvf jdk-8u251-linux-x64.tar.gz -C /opt/

sudo vi /etc/profile

# 追加内容
JAVA_HOME=/opt/jdk1.8.0_251
export PATH=$JAVA_HOME/bin:$PATH

source /etc/profile

# java -version
java version "1.8.0_251"
Java(TM) SE Runtime Environment (build 1.8.0_251-b08)
Java HotSpot(TM) 64-Bit Server VM (build 25.251-b08, mixed mode)

安装编译基本软件
```
yum install -y gcc-c++ rpm-build git
```

安装 Maven 3.6.3

tar -zxvf apache-maven-3.6.3-bin.tar.gz -C /opt/

vi /etc/profile

MAVEN_HOME=/opt/apache-maven-3.6.3
export PATH=$MAVEN_HOME/bin:$PATH

source /etc/profile
mvn -version

通常还可以在 setttings.xml 配置阿里云镜像（但是后面某一步骤的时候会报错，还是得注释掉切回中央仓库，有时候还是阿里镜像快，总之遇到下载问题，来回切换试试）

<mirror>
    <id>aliyunmaven</id>
    <mirrorOf>*</mirrorOf>
    <name>阿里云公共仓库</name>
    <url>https://maven.aliyun.com/repository/public</url>
</mirror>

Python 2.7（CentOS7自带）

安装 setuptools

yum install -y wget
wget https://pypi.python.org/packages/2.7/s/setuptools/setuptools-0.6c11-py2.7.egg#md5=fe1f997bc722265116870bc7919059ea
sh setuptools-0.6c11-py2.7.egg

安装 nodejs 环境

yum install epel-release -y
yum install nodejs -y
# 升级 nodejs
sudo npm install n -g
sudo n stable

发现没有 npm 命令，切换源，再次安装

curl --silent --location https://rpm.nodesource.com/setup_10.x | bash - 
yum install -y nodejs 
# 使用淘宝镜像
npm install -g cnpm --registry=https://registry.npm.taobao.org

查看版本

[root@localhost ~]# node -v
v10.24.1
[root@localhost ~]# npm -v
6.14.12

部署 apache https（后面编译大包需要提前下载并替换URL，详见 3.6）

安装 yum 相关工具

yum install yum-utils -y
yum repolist
yum install createrepo -y

安装 httpd 服务

yum install httpd -y

启动、查看服务状态

systemctl start httpd
systemctl status httpd

设置开机自启

systemctl enable httpd

关闭防火墙（防止访问不了服务）、查看状态、永久关闭

systemctl stop firewalld.service
systemctl status firewalld.service
systemctl disable firewalld.service

访问该主机IP

http://192.168.xxx.xxx/

配置GitHub hosts（没试过，不过确实GitHub不好访问）
```
vim /etc/hosts
140.82.113.4 github.com
```

二、下载编译

官方编译文档地址：cwiki.apache.org/confluence/…

wget https://archive.apache.org/dist/ambari/ambari-2.7.5/apache-ambari-2.7.5-src.tar.gz
tar xfvz apache-ambari-2.7.5-src.tar.gz
cd apache-ambari-2.7.5-src
mvn versions:set -DnewVersion=2.7.5.0.0
 
pushd ambari-metrics
mvn versions:set -DnewVersion=2.7.5.0.0
popd

由于后续是部署到CentOS系统上，所以执行以下命令：(比官网多了一个-Drat.skip 来跳过文件许可，不然会报错，错误详见 3.5)

mvn -B clean install rpm:rpm -DnewVersion=2.7.5.0.0 -DbuildNumber=5895e4ed6b30a2da8a90fee2403b6cab91d19972 -DskipTests -Dpython.ver="python >= 2.6" -Drat.skip

编译过程中可能会遇到很多报错，参考 “三、编译过程异常处理” 解决。当解决完后，可以继续编译，不用从头编译，就像编译web模块时候报错，提示中的：
mvn <args> -rf :ambari-web
即，解决完后可以直接运行：
mvn -B clean install rpm:rpm -DnewVersion=2.7.5.0.0 -DbuildNumber=5895e4ed6b30a2da8a90fee2403b6cab91d19972 -DskipTests -Dpython.ver="python >= 2.6" -Drat.skip -rf :ambari-web

最终编译完成，显示如下结果：

[INFO] ------------------------------------------------------------------------
[INFO] Reactor Summary:
[INFO] 
[INFO] Ambari Main 2.7.5.0.0 .............................. SUCCESS [  3.742 s]
[INFO] Apache Ambari Project POM 2.7.5.0.0 ................ SUCCESS [  0.215 s]
[INFO] Ambari Web 2.7.5.0.0 ............................... SUCCESS [01:15 min]
[INFO] Ambari Views 2.7.5.0.0 ............................. SUCCESS [ 10.204 s]
[INFO] Ambari Admin View 2.7.5.0.0 ........................ SUCCESS [01:12 min]
[INFO] ambari-utility 1.0.0.0-SNAPSHOT .................... SUCCESS [01:17 min]
[INFO] ambari-metrics 2.7.5.0.0 ........................... SUCCESS [  0.809 s]
[INFO] Ambari Metrics Common 2.7.5.0.0 .................... SUCCESS [  9.959 s]
[INFO] Ambari Metrics Hadoop Sink 2.7.5.0.0 ............... SUCCESS [ 15.259 s]
[INFO] Ambari Metrics Flume Sink 2.7.5.0.0 ................ SUCCESS [  2.854 s]
[INFO] Ambari Metrics Kafka Sink 2.7.5.0.0 ................ SUCCESS [  6.413 s]
[INFO] Ambari Metrics Storm Sink 2.7.5.0.0 ................ SUCCESS [  3.781 s]
[INFO] Ambari Metrics Storm Sink (Legacy) 2.7.5.0.0 ....... SUCCESS [  3.524 s]
[INFO] Ambari Metrics Collector 2.7.5.0.0 ................. SUCCESS [05:35 min]
[INFO] Ambari Metrics Monitor 2.7.5.0.0 ................... SUCCESS [  1.619 s]
[INFO] Ambari Metrics Grafana 2.7.5.0.0 ................... SUCCESS [  2.412 s]
[INFO] Ambari Metrics Host Aggregator 2.7.5.0.0 ........... SUCCESS [ 10.956 s]
[INFO] Ambari Metrics Assembly 2.7.5.0.0 .................. SUCCESS [01:57 min]
[INFO] Ambari Service Advisor 1.0.0.0-SNAPSHOT ............ SUCCESS [  2.854 s]
[INFO] Ambari Server 2.7.5.0.0 ............................ SUCCESS [05:54 min]
[INFO] Ambari Functional Tests 2.7.5.0.0 .................. SUCCESS [  1.047 s]
[INFO] Ambari Agent 2.7.5.0.0 ............................. SUCCESS [01:41 min]
[INFO] ambari-logsearch 2.7.5.0.0 ......................... SUCCESS [  1.041 s]
[INFO] Ambari Logsearch Appender 2.7.5.0.0 ................ SUCCESS [  0.722 s]
[INFO] Ambari Logsearch Config Api 2.7.5.0.0 .............. SUCCESS [  0.533 s]
[INFO] Ambari Logsearch Config JSON 2.7.5.0.0 ............. SUCCESS [  0.378 s]
[INFO] Ambari Logsearch Config Solr 2.7.5.0.0 ............. SUCCESS [  0.471 s]
[INFO] Ambari Logsearch Config Zookeeper 2.7.5.0.0 ........ SUCCESS [  0.394 s]
[INFO] Ambari Logsearch Config Local 2.7.5.0.0 ............ SUCCESS [  0.449 s]
[INFO] Ambari Logsearch Log Feeder Plugin Api 2.7.5.0.0 ... SUCCESS [  0.490 s]
[INFO] Ambari Logsearch Log Feeder Container Registry 2.7.5.0.0 SUCCESS [  0.641 s]
[INFO] Ambari Logsearch Log Feeder 2.7.5.0.0 .............. SUCCESS [  5.773 s]
[INFO] Ambari Logsearch Web 2.7.5.0.0 ..................... SUCCESS [01:20 min]
[INFO] Ambari Logsearch Server 2.7.5.0.0 .................. SUCCESS [  8.560 s]
[INFO] Ambari Logsearch Assembly 2.7.5.0.0 ................ SUCCESS [  0.410 s]
[INFO] Ambari Logsearch Integration Test 2.7.5.0.0 ........ SUCCESS [  1.639 s]
[INFO] ambari-infra 2.7.5.0.0 ............................. SUCCESS [  0.450 s]
[INFO] Ambari Infra Solr Client 2.7.5.0.0 ................. SUCCESS [  5.193 s]
[INFO] Ambari Infra Solr Plugin 2.7.5.0.0 ................. SUCCESS [  1.112 s]
[INFO] Ambari Infra Manager 2.7.5.0.0 ..................... SUCCESS [  8.860 s]
[INFO] Ambari Infra Assembly 2.7.5.0.0 .................... SUCCESS [  0.299 s]
[INFO] Ambari Infra Manager Integration Tests 2.7.5.0.0 ... SUCCESS [  3.329 s]
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time:  22:12 min
[INFO] Finished at: 2022-10-19T11:01:57+08:00
[INFO] ------------------------------------------------------------------------

编译后的各个模块的 rpms 在对应的路径下，如：

apache-ambari-2.7.5-src/ambari-server/target/rpm/ambari-server/RPMS/x86_64/ambari-server-2.7.5.0-0.x86_64.rpm
apache-ambari-2.7.5-src/ambari-agent/target/rpm/ambari-agent/RPMS/x86_64/ambari-agent-2.7.5.0-0.x86_64.rpm

同时提供一张，可以直接用 yum 源的 rpm 包，来做对比。由于还没实际用过自己编译的，再次推测只要替换掉对应的包就能进行安装、升级：

三、编译过程异常处理

3.1 Could not extract archive: '/root/.m2/repository/xxxxxx'

[ERROR] Failed to execute goal com.github.eirslett:frontend-maven-plugin:1.4:install-node-and-yarn (install node and yarn) on project ambari-web: Could not extract the Yarn archive: Could not extract archive: '/root/.m2/repository/com/github/eirslett/yarn/0.23.2/yarn-0.23.2./yarn-v0.23.2.tar.gz': EOFException -> [Help 1]

解决方案

有可能是因为网络问题包没有下载完整，需要手动下载该包放到本地仓库，或者多次重试。【其他类似错误相同处理】
curl -o /root/.m2/repository/com/github/eirslett/yarn/0.23.2/yarn-0.23.2./yarn-v0.23.2.tar.gz https://github.com/yarnpkg/yarn/releases/download/v0.23.2/yarn-v0.23.2.tar.gz

3.2 The platform "linux" is incompatible with this module.

[ERROR] warning fsevents@0.3.8: The platform "linux" is incompatible with this module.

解决方案

npm install pm2@latest -g
pm2 install pm2-slack

3.3 Failed to run task: 'yarn install --ignore-engines --pure-lockfile' failed.

[ERROR] Failed to execute goal com.github.eirslett:frontend-maven-plugin:1.4:yarn (yarn install) on project ambari-web: Failed to run task: 'yarn install --ignore-engines --pure-lockfile' failed. org.apache.commons.exec.ExecuteException: Process exited with an error: 1 (Exit value: 1) -> [Help 1]

解决方案

网络环境不好，nodejs会通过 yarn去加载依赖包，多试几次。【网上是这么说的，但是实际上操作的时候，是去执行了1.1中的第六步，安装nodejs，还是报错，然后又看到了3.2中的异常，3.2修复后，这个问题就消失了】

3.4 Bower install

[ERROR] Failed to execute goal org.codehaus.mojo:exec-maven-plugin:1.2.1:exec (Bower install) on project ambari-admin: Command execution failed. Process exited with an error: 1 (Exit value: 1) -> [Help 1]

解决方案（网上找的，都没用，因为不是对应的错误，但也做个记录）

方式一：【没有成功】

cd ambari-admin/src/main/resources/ui/admin-web目录下，编辑 .bowerrc ，修改后的内容如下：
{
“directory”: “app/bower_components”,
“allow_root”: true
}
个人将ambari-admin（包括子目录ambari-admin/src/main/resources/ui/admin-web）里面生成的node、node_modules删除，保持干净的源码环境，再单独编译ambari-admin
find ~/.m2/repository/ -name “*.lastUpdated” -exec rm -rf {} ;

方式二：【还是失败】

cd ambari-admin/src/main/resources/ui/admin-web/
npm install -g bower
# 中间可能会安装失败，多试几次毕竟下载的是github的。
#（我一直不行，最后下载代理后可以了。https://github.com/dotnetcore/FastGithub）
bower install --allow-root

最终还是不行，然后我往上看，发现还有其他报错：

[INFO] --- exec-maven-plugin:1.2.1:exec (Bower install) @ ambari-admin ---
bower                            error Unexpected token {

Stack trace:
SyntaxError: Unexpected token {
    at exports.runInThisContext (vm.js:53:16)
    at Module._compile (module.js:373:25)
    at Object.Module._extensions..js (module.js:416:10)
    at Module.load (module.js:343:32)
    at Function.Module._load (module.js:300:12)
    at Module.require (module.js:353:17)
    at require (internal/module.js:12:17)
    at Object.<anonymous> (/root/src_code/ambari/apache-ambari-2.7.5-src/ambari-admin/src/main/resources/ui/admin-web/node_modules/bower/node_modules/bower-registry-client/node_modules/request/lib/cookies.js:3:13)
    at Module._compile (module.js:409:26)
    at Object.Module._extensions..js (module.js:416:10)

最终解决方案

vi ambari-admin/pom.xml

<argument>${basedir}/src/main/resources/ui/admin-web/node_modules/bower/bin/bower</argument>
改为
<argument>bower</argument>

3.5 Too many files with unapproved license

[ERROR] Failed to execute goal org.apache.rat:apache-rat-plugin:0.12:check (default) on project ambari-admin: Too many files with unapproved license: 1 See RAT report in: /root/src_code/ambari/apache-ambari-2.7.5-src/ambari-admin/target/rat.txt -> [Help 1]

解决方案

mvn 命令后加上参数 -Drat.skip 来绕过文件许可。

3.6 Can't get `https://s3.amazonaws.com/xxx`

[ERROR] Failed to execute goal org.apache.maven.plugins:maven-antrun-plugin:1.7:run (default) on project ambari-metrics-timelineservice: An Ant BuildException has occured: Can't get https://s3.amazonaws.com/dev.hortonworks.com/HDP/centos7/3.x/BUILDS/3.1.4.0-315/tars/hbase/hbase-2.0.2.3.1.4.0-315-bin.tar.gz to /root/src_code/ambari/apache-ambari-2.7.5-src/ambari-metrics/ambari-metrics-timelineservice/target/embedded/hbase.tar.gz
[ERROR] around Ant part ...<get usetimestamp="true" src="https://s3.amazonaws.com/dev.hortonworks.com/HDP/centos7/3.x/BUILDS/3.1.4.0-315/tars/hbase/hbase-2.0.2.3.1.4.0-315-bin.tar.gz" dest="/root/src_code/ambari/apache-ambari-2.7.5-src/ambari-metrics/ambari-metrics-timelineservice/target/embedded/hbase.tar.gz"/>... @ 5:289 in /root/src_code/ambari/apache-ambari-2.7.5-src/ambari-metrics/ambari-metrics-timelineservice/target/antrun/build-Download HBase.xml

解决方案

提前下载好包，用下载工具下载下来（PS：迅雷可以直接镜像加速），再部署到自己本地的web服务器上（可以参考1.1中第7点），再修改这个pom.xml中的url地址。

需要下载的大包：

https://s3.amazonaws.com/dev.hortonworks.com/HDP/centos7/3.x/BUILDS/3.1.4.0-315/tars/hbase/hbase-2.0.2.3.1.4.0-315-bin.tar.gz
https://s3.amazonaws.com/dev.hortonworks.com/HDP/centos7/3.x/BUILDS/3.1.4.0-315/tars/hadoop/hadoop-3.1.1.3.1.4.0-315.tar.gz
https://dl.grafana.com/oss/release/grafana-6.4.2.linux-amd64.tar.gz
https://s3.amazonaws.com/dev.hortonworks.com/HDP/centos7/3.x/BUILDS/3.1.4.0-315/tars/phoenix/phoenix-5.0.0.3.1.4.0-315.tar.gz

放入https服务目录下：（我是放在 /var/www/html/tars/ 下，然后访问网页 http://192.168.xxx.xxx/tars/，就会出现如图所示）

需要修改的URL：vi ambari-metrics/pom.xml

<hbase.tar>https://s3.amazonaws.com/dev.hortonworks.com/HDP/centos7/3.x/BUILDS/3.1.4.0-315/tars/hbase/hbase-2.0.2.3.1.4.0-315-bin.tar.gz</hbase.tar>
<hbase.folder>hbase-2.0.2.3.1.4.0-315</hbase.folder>
<hadoop.tar>https://s3.amazonaws.com/dev.hortonworks.com/HDP/centos7/3.x/BUILDS/3.1.4.0-315/tars/hadoop/hadoop-3.1.1.3.1.4.0-315.tar.gz</hadoop.tar>
<hadoop.folder>hadoop-3.1.1.3.1.4.0-315</hadoop.folder>
<grafana.folder>grafana-6.4.2</grafana.folder>
<grafana.tar>https://dl.grafana.com/oss/release/grafana-6.4.2.linux-amd64.tar.gz</grafana.tar>
<phoenix.tar>https://s3.amazonaws.com/dev.hortonworks.com/HDP/centos7/3.x/BUILDS/3.1.4.0-315/tars/phoenix/phoenix-5.0.0.3.1.4.0-315.tar.gz</phoenix.tar>
<phoenix.folder>phoenix-5.0.0.3.1.4.0-315</phoenix.folder>

改成：

<hbase.tar>http://192.168.xxx.xxx/tars/hbase-2.0.2.3.1.4.0-315-bin.tar.gz</hbase.tar>
<hbase.folder>hbase-2.0.2.3.1.4.0-315</hbase.folder>
<hadoop.tar>http://192.168.xxx.xxx/tars/hadoop-3.1.1.3.1.4.0-315.tar.gz</hadoop.tar>
<hadoop.folder>hadoop-3.1.1.3.1.4.0-315</hadoop.folder>
<grafana.folder>grafana-6.4.2</grafana.folder>
<grafana.tar>http://192.168.xxx.xxx/tars/grafana-6.4.2.linux-amd64.tar.gz</grafana.tar>
<phoenix.tar>http://192.168.xxx.xxx/tars/phoenix-5.0.0.3.1.4.0-315.tar.gz</phoenix.tar>
<phoenix.folder>phoenix-5.0.0.3.1.4.0-315</phoenix.folder>

3.7 Python.h: No such file or directory

[exec] psutil/_psutil_linux.c:12:20: fatal error: Python.h: No such file or directory
[exec]  #include <Python.h>
[exec]                     ^
[exec] compilation terminated.
[exec] error: command 'gcc' failed with exit status 1

[ERROR] Failed to execute goal org.apache.maven.plugins:maven-antrun-plugin:1.7:run (psutils-compile) on project ambari-metrics-host-monitoring: An Ant BuildException has occured: exec returned: 1
[ERROR] around Ant part ...<exec failonerror="true" dir="/root/src_code/ambari/apache-ambari-2.7.5-src/ambari-metrics/ambari-metrics-host-monitoring/src/main/python/psutil" executable="/root/src_code/ambari/apache-ambari-2.7.5-src/ambari-metrics/ambari-metrics-host-monitoring/../../ambari-common/src/main/unix/ambari-python-wrap">... @ 4:307 in /root/src_code/ambari/apache-ambari-2.7.5-src/ambari-metrics/ambari-metrics-host-monitoring/target/antrun/build-psutils-compile.xml

解决方案

yum install -y python-devel

3.8 PhantomJS not found on PATH

[ERROR] Output:
[ERROR] PhantomJS not found on PATH
[ERROR] Downloading https://github.com/Medium/phantomjs/releases/download/v2.1.1/phantomjs-2.1.1-linux-x86_64.tar.bz2
[ERROR] Saving to /tmp/logsearch_npm_config_tmp/phantomjs/phantomjs-2.1.1-linux-x86_64.tar.bz2
[ERROR] Receiving...
[ERROR] 
[ERROR] Error making request.
[ERROR] Error: tunneling socket could not be established, cause=connect ECONNREFUSED 127.0.0.1:38457
[ERROR]     at ClientRequest.onError (/root/src_code/ambari/apache-ambari-2.7.5-src/ambari-logsearch/ambari-logsearch-web/target/webapp-build/node_modules/tunnel-agent/index.js:177:17)
[ERROR]     at Object.onceWrapper (events.js:316:30)
[ERROR]     at emitOne (events.js:115:13)
[ERROR]     at ClientRequest.emit (events.js:210:7)
[ERROR]     at Socket.socketErrorListener (_http_client.js:401:9)
[ERROR]     at emitOne (events.js:115:13)
[ERROR]     at Socket.emit (events.js:210:7)
[ERROR]     at emitErrorNT (internal/streams/destroy.js:64:8)
[ERROR]     at _combinedTickCallback (internal/process/next_tick.js:138:11)
[ERROR]     at process._tickCallback (internal/process/next_tick.js:180:9)

解决方案

一般都是网络问题，可以喝之前一样先下载好放到对应目录下：(根据报错信息可以发现)

需要下载的文件
https://github.com/Medium/phantomjs/releases/download/v2.1.1/phantomjs-2.1.1-linux-x86_64.tar.bz2
下载完放入
/tmp/logsearch_npm_config_tmp/phantomjs/phantomjs-2.1.1-linux-x86_64.tar.bz2

Ambari 2.7.5 源码编译

一、编译准备

二、下载编译

三、编译过程异常处理

3.1 Could not extract archive: '/root/.m2/repository/xxxxxx'

3.2 The platform "linux" is incompatible with this module.

3.3 Failed to run task: 'yarn install --ignore-engines --pure-lockfile' failed.

3.4 Bower install

3.5 Too many files with unapproved license

3.6 Can't get https://s3.amazonaws.com/xxx

3.7 Python.h: No such file or directory

3.8 PhantomJS not found on PATH

3.6 Can't get `https://s3.amazonaws.com/xxx`