Flink job在IDEA上正常运行,部署到服务器上报Caused by: java.lang.ClassNotFoundException

2,271 阅读1分钟

分析原因:

  • 缺少jar包
  • jar包中没有这个类。运行Flink包报这个错误,查看打的jar包中确实有这个类,但是运行的时候就是找不到,很奇怪。经查发现,Flink程序在运行时,会先加载自己lib下的类,然后再加载用户jar中的类,发现lib中有同名的类路径,但是没有类,如flink lib中有这个org.apache.hadoop.fs路径,但是没有FileSystem类,但是运行时需要这个类,即使后面的jar包中有org.apache.hadoop.fs.FileSystem也不去加载。
  • 打包方式有问题,打包路径有问题 flink打包配置:
<build>
    <plugins>
        <plugin>
            <groupId>org.apache.maven.plugins</groupId>
            <artifactId>maven-shade-plugin</artifactId>
            <version>3.0.0</version>
            <executions>
                <execution>
                    <phase>package</phase>
                    <goals>
                        <goal>shade</goal>
                    </goals>
                    <configuration>
                        <createDependencyReducedPom>false</createDependencyReducedPom>
                        <artifactSet>
                            <excludes>
                                <exclude>com.google.code.findbugs:jsr305</exclude>
                                <exclude>org.slf4j:*</exclude>
                                <exclude>log4j:*</exclude>
                            </excludes>
                        </artifactSet>
                        <filters>
                            <filter>
                                <!-- Do not copy the signatures in the META-INF folder.
                                Otherwise, this might cause SecurityExceptions when using the JAR. -->
                                <artifact>*:*</artifact>
                                <excludes>
                                    <exclude>META-INF/*.SF</exclude>
                                    <exclude>META-INF/*.DSA</exclude>
                                    <exclude>META-INF/*.RSA</exclude>
                                </excludes>
                            </filter>
                        </filters>
                        <transformers>
                            <transformer
                                    implementation="org.apache.maven.plugins.shade.resource.ManifestResourceTransformer">
                                <mainClass>cn.indusec.analysis.dataprocess.DataProcessJob</mainClass>
                            </transformer>
                        </transformers>
                    </configuration>
                </execution>
            </executions>
        </plugin>
    </plugins>
</build>