本文已参与「新人创作礼」活动,一起开启掘金创作之路。
Azkaban 安装部署
背景
截止 ambari HDP2.6.5官方还没有支持Azkaban的的安装,但是项目中用到了,为了方便管理,将Azkaban放入ambari管理
Azkaban编译
从源代码构建 Azkaban构建使用Gradle(使用Gradle包装器gradlew运行时自动下载),并且需要Java 8或更高版本,还需要有git。
下面的命令上运行*如Linux,OS X. nix平台
#Install Git
#yum -y install git
# Build Azkaban
./gradlew build
# Clean the build
./gradlew clean
# Build and install distributions
./gradlew installDist
# Run tests
./gradlew test
# Build without running tests
./gradlew build -x test
这些都是标准的Gradle命令。请查看Gradle文档以获取更多信息。
Gradle在项目目录中创建.tar.gz文件。例如。./azkaban-solo-server/build/distributions/azkaban-solo-server-0.1.0-SNAPSHOT.tar.gz。
官网说明
问题
./gradlew build
提示:
Could not determine the dependencies of task ':az-flow-trigger-dependency-type:kafka-event-trigger:fatJar'.
> Could not resolve all files for configuration ':az-flow-trigger-dependency-type:kafka-event-trigger:compile'
> Could not download kafka_2.11.jar (org.apache.kafka:kafka_2.11:0.10.2.0)
> Could not get resource 'https://repo.maven.apache.org/maven2/org/apache/kafka/kafka_2.11/0.10.2.0/kafka_2.11-0.10.2.0.jar'
后面省略n个错误
解决
解决:我们需要在/az-flow-trigger-dependency-type/kafka-event-trigger目录运行。
# Build Azkaban
../../gradlew build
# Clean the build
../../gradlew clean
# Build without running tests
../../gradlew build -x test
安装
将编译好的的放在自己建的yum源
#将编译好的包复制到yum源目录
cp azkaban-exec-server-0.1.0-SNAPSHOT.tar.gz /var/www/html/Package/
cp azkaban-web-server-0.1.0-SNAPSHOT.tar.gz /var/www/html/Package/
tar -zxvf azkaban-db-0.1.0-SNAPSHOT.tar.gz
cp create-all-sql-0.1.0-SNAPSHOT.sql /var/www/html/Package/
cd /var/www/html/Package/
#createrepo
createrepo .
构建ambari server
VERSION=`hdp-select status hadoop-client | sed 's/hadoop-client - \([0-9]\.[0-9]\).*/\1/'`
sudo git clone https://github.com/jzyhappy/ambari-azkaban-service.git /var/lib/ambari-server/resources/stacks/HDP/$VERSION/services/AZKABAN
重启ambari
ambari-server restart