编译Flink1.11保姆级教程及解决遇到的坑

1,891 阅读1分钟

在Maven仓库中添加镜像:

<mirror>
    <id>nexus-aliyun</id>
    <mirrorOf>*,!jeecg,!jeecg-snapshots,!mapr-releases</mirrorOf>
    <name>Nexus aliyun</name>
    <url>http://maven.aliyun.com/nexus/content/groups/public</url>
</mirror>

<mirror>
    <id>mapr-public</id>
    <mirrorOf>mapr-releases</mirrorOf>
    <name>mapr-releases</name>
    <url>https://maven.aliyun.com/repository/mapr-public</url>
</mirror>

<mirror>
  <id>confluent</id>
  <mirrorOf>confluent</mirrorOf>
  <name>confluent</name>
  <url>http://packages.confluent.io/maven/</url>
</mirror>

拉取Flink1.11 release的分支,然后cd到Flink文件夹下,使用编译命令:

mvn clean package -T 4 -Dfast -Dmaven.compile.fork=true -DskipTests -Dscala-2.11

命令具体解释如下:

mvn clean install \
	-Dfast \ # 跳过QA 的插件和 JavaDocs 的生成
	-T 4 \ # 支持多处理器或者处理器核数参数,加快构建速度,推荐Maven3.3及以上
	-Dmaven.compile.fork=true #允许多线程编译,推荐maven在3.3及以上
    	-DskipTests \ # 跳过测试部分

一. flink-runtime-web

其中,一直编译失败的是flink-runtime-web模块,故先cd flink-runtime-web进行单独编译。

坑1:flink-test-utils-junit:1.11-SNAPSHOT flink-test-utils_2.11:1.11-SNAPSHOT依赖一直下不下来,于是我把version为1.11-SNAPSHOT直接改为1.11.1,通过!

  <dependency>
      <groupId>org.apache.flink</groupId>
      <artifactId>flink-test-utils-junit</artifactId>
      <!--新增-->
      <version>1.11.1</version>
      <scope>test</scope>
  </dependency>

  <dependency>
      <groupId>org.apache.flink</groupId>
      <artifactId>flink-test-utils_${scala.binary.version}</artifactId>
      <!--修改-->
      <version>1.11.1</version>
      <scope>test</scope>
  </dependency>

坑2:编译卡在 Running 'npm ci –cache-max=0 –no-save'flink 。因为编译中需要安装node包,于是修改flink-runtime-web/pom.xml:

<execution>
  <id>install node and npm</id>
  <goals>
      <goal>install-node-and-npm</goal>
  </goals>
  <configuration>
      <!--新增-->
      <nodeDownloadRoot>https://registry.npm.taobao.org/dist/</nodeDownloadRoot>
      <npmDownloadRoot>https://registry.npmjs.org/npm/</npmDownloadRoot>
      <nodeVersion>v10.9.0</nodeVersion>
  </configuration>
</execution>
<execution>
  <id>npm install</id>
  <goals>
      <goal>npm</goal>
  </goals>
  <configuration>
      <!--修改-->
      <arguments>install -registry=https://registry.npm.taobao.org --cache-max=0 --no-save</arguments>
      <environmentVariables>
          <HUSKY_SKIP_INSTALL>true</HUSKY_SKIP_INSTALL>
      </environmentVariables>
  </configuration>
</execution>
<execution>
  <id>npm run build</id>
  <goals>
      <goal>npm</goal>
  </goals>
  <configuration>
      <arguments>run build</arguments>
      <!--新增-->
      <npmRegistryURL>https://registry.npm.taobao.org</npmRegistryURL>
  </configuration>
</execution>

如果web-dashboard/node_modules中有缓存,建议删掉重新下载,避免混乱。 成功!