flink 1.16.3 代码提交任务报错,报错信息:
Caused by: java.lang.RuntimeException: org.apache.flink.runtime.JobException: Cannot instantiate the coordinator for operator Source: source_test[1] -> DropUpdateBefore[2] -> ConstraintEnforcer[3] -> Sink: sink_test[3]
at org.apache.flink.util.ExceptionUtils.rethrow(ExceptionUtils.java:321)
at org.apache.flink.util.function.FunctionUtils.lambda$uncheckedSupplier$4(FunctionUtils.java:114)
at java.util.concurrent.CompletableFuture$AsyncSupply.run(CompletableFuture.java:1590)
... 3 more
Caused by: org.apache.flink.runtime.JobException: Cannot instantiate the coordinator for operator Source: source_test[1] -> DropUpdateBefore[2] -> ConstraintEnforcer[3] -> Sink: sink_test[3]
at org.apache.flink.runtime.executiongraph.ExecutionJobVertex.initialize(ExecutionJobVertex.java:229)
at org.apache.flink.runtime.executiongraph.DefaultExecutionGraph.initializeJobVertex(DefaultExecutionGraph.java:901)
at org.apache.flink.runtime.executiongraph.DefaultExecutionGraph.initializeJobVertices(DefaultExecutionGraph.java:891)
at org.apache.flink.runtime.executiongraph.DefaultExecutionGraph.attachJobGraph(DefaultExecutionGraph.java:848)
at org.apache.flink.runtime.executiongraph.DefaultExecutionGraph.attachJobGraph(DefaultExecutionGraph.java:830)
at org.apache.flink.runtime.executiongraph.DefaultExecutionGraphBuilder.buildGraph(DefaultExecutionGraphBuilder.java:203)
at org.apache.flink.runtime.scheduler.DefaultExecutionGraphFactory.createAndRestoreExecutionGraph(DefaultExecutionGraphFactory.java:156)
at org.apache.flink.runtime.scheduler.SchedulerBase.createAndRestoreExecutionGraph(SchedulerBase.java:361)
at org.apache.flink.runtime.scheduler.SchedulerBase.<init>(SchedulerBase.java:206)
at org.apache.flink.runtime.scheduler.DefaultScheduler.<init>(DefaultScheduler.java:134)
at org.apache.flink.runtime.scheduler.DefaultSchedulerFactory.createInstance(DefaultSchedulerFactory.java:152)
at org.apache.flink.runtime.jobmaster.DefaultSlotPoolServiceSchedulerFactory.createScheduler(DefaultSlotPoolServiceSchedulerFactory.java:119)
at org.apache.flink.runtime.jobmaster.JobMaster.createScheduler(JobMaster.java:369)
at org.apache.flink.runtime.jobmaster.JobMaster.<init>(JobMaster.java:346)
at org.apache.flink.runtime.jobmaster.factories.DefaultJobMasterServiceFactory.internalCreateJobMasterService(DefaultJobMasterServiceFactory.java:123)
at org.apache.flink.runtime.jobmaster.factories.DefaultJobMasterServiceFactory.lambda$createJobMasterService$0(DefaultJobMasterServiceFactory.java:95)
at org.apache.flink.util.function.FunctionUtils.lambda$uncheckedSupplier$4(FunctionUtils.java:112)
... 4 more
Caused by: java.io.InvalidClassException: com.ververica.cdc.connectors.shaded.com.fasterxml.jackson.databind.cfg.MapperConfig; incompatible types for field _mapperFeatures
at java.io.ObjectStreamClass.matchFields(ObjectStreamClass.java:2299)
at java.io.ObjectStreamClass.getReflector(ObjectStreamClass.java:2193)
at java.io.ObjectStreamClass.initNonProxy(ObjectStreamClass.java:669)
at java.io.ObjectInputStream.readNonProxyDesc(ObjectInputStream.java:1843)
at java.io.ObjectInputStream.readClassDesc(ObjectInputStream.java:1713)
at java.io.ObjectInputStream.readNonProxyDesc(ObjectInputStream.java:1843)
at java.io.ObjectInputStream.readClassDesc(ObjectInputStream.java:1713)
at java.io.ObjectInputStream.readNonProxyDesc(ObjectInputStream.java:1843)
at java.io.ObjectInputStream.readClassDesc(ObjectInputStream.java:1713)
at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:2000)
at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1535)
at java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:2245)
at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:2169)
at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:2027)
at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1535)
at java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:2245)
at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:2169)
at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:2027)
at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1535)
at java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:2245)
at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:2169)
at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:2027)
at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1535)
at java.io.ObjectInputStream.readArray(ObjectInputStream.java:1933)
at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1529)
at java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:2245)
at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:2169)
at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:2027)
at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1535)
at java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:2245)
at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:2169)
at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:2027)
at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1535)
at java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:2245)
at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:2169)
at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:2027)
at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1535)
at java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:2245)
at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:2169)
at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:2027)
at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1535)
at java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:2245)
at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:2169)
at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:2027)
at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1535)
at java.io.ObjectInputStream.readObject(ObjectInputStream.java:422)
at org.apache.flink.util.InstantiationUtil.deserializeObject(InstantiationUtil.java:617)
at org.apache.flink.util.InstantiationUtil.deserializeObject(InstantiationUtil.java:602)
at org.apache.flink.util.InstantiationUtil.deserializeObject(InstantiationUtil.java:589)
at org.apache.flink.util.SerializedValue.deserializeValue(SerializedValue.java:67)
at org.apache.flink.runtime.operators.coordination.OperatorCoordinatorHolder.create(OperatorCoordinatorHolder.java:488)
at org.apache.flink.runtime.executiongraph.ExecutionJobVertex.createOperatorCoordinatorHolder(ExecutionJobVertex.java:286)
at org.apache.flink.runtime.executiongraph.ExecutionJobVertex.initialize(ExecutionJobVertex.java:223)
... 20 more
2024-08-06 17:06:13,522 INFO org.apache.flink.runtime.dispatcher.StandaloneDispatcher [] - Could not store completed job Flink Demo - To sink_test(5e97c8b468981fcb1d2efcb37c6996a5).
java.io.FileNotFoundException: /tmp/executionGraphStore-773d174a-17d0-4be1-bccf-a7f8d3f90f87/5e97c8b468981fcb1d2efcb37c6996a5 (没有那个文件或目录)
at java.io.FileOutputStream.open0(Native Method) ~[?:1.8.0_141]
at java.io.FileOutputStream.open(FileOutputStream.java:270) ~[?:1.8.0_141]
at java.io.FileOutputStream.<init>(FileOutputStream.java:213) ~[?:1.8.0_141]
at java.io.FileOutputStream.<init>(FileOutputStream.java:162) ~[?:1.8.0_141]
at org.apache.flink.runtime.dispatcher.FileExecutionGraphInfoStore.storeExecutionGraphInfo(FileExecutionGraphInfoStore.java:281) ~[flink-dist-1.16.3.jar:1.16.3]
at org.apache.flink.runtime.dispatcher.FileExecutionGraphInfoStore.put(FileExecutionGraphInfoStore.java:203) ~[flink-dist-1.16.3.jar:1.16.3]
at org.apache.flink.runtime.dispatcher.Dispatcher.writeToExecutionGraphInfoStore(Dispatcher.java:1185) ~[flink-dist-1.16.3.jar:1.16.3]
at org.apache.flink.runtime.dispatcher.Dispatcher.jobReachedTerminalState(Dispatcher.java:1115) ~[flink-dist-1.16.3.jar:1.16.3]
at org.apache.flink.runtime.dispatcher.Dispatcher.handleJobManagerRunnerResult(Dispatcher.java:675) ~[flink-dist-1.16.3.jar:1.16.3]
at org.apache.flink.runtime.dispatcher.Dispatcher.lambda$runJob$5(Dispatcher.java:625) ~[flink-dist-1.16.3.jar:1.16.3]
at java.util.concurrent.CompletableFuture.uniHandle(CompletableFuture.java:822) ~[?:1.8.0_141]
at java.util.concurrent.CompletableFuture$UniHandle.tryFire(CompletableFuture.java:797) ~[?:1.8.0_141]
at java.util.concurrent.CompletableFuture$Completion.run(CompletableFuture.java:442) ~[?:1.8.0_141]
at org.apache.flink.runtime.rpc.akka.AkkaRpcActor.lambda$handleRunAsync$4(AkkaRpcActor.java:453) ~[flink-rpc-akka_c0c89ca0-3661-4f02-afd5-3d51bafc7d6a.jar:1.16.3]
at org.apache.flink.runtime.concurrent.akka.ClassLoadingUtils.runWithContextClassLoader(ClassLoadingUtils.java:68) ~[flink-rpc-akka_c0c89ca0-3661-4f02-afd5-3d51bafc7d6a.jar:1.16.3]
at org.apache.flink.runtime.rpc.akka.AkkaRpcActor.handleRunAsync(AkkaRpcActor.java:453) ~[flink-rpc-akka_c0c89ca0-3661-4f02-afd5-3d51bafc7d6a.jar:1.16.3]
at org.apache.flink.runtime.rpc.akka.AkkaRpcActor.handleRpcMessage(AkkaRpcActor.java:218) ~[flink-rpc-akka_c0c89ca0-3661-4f02-afd5-3d51bafc7d6a.jar:1.16.3]
at org.apache.flink.runtime.rpc.akka.FencedAkkaRpcActor.handleRpcMessage(FencedAkkaRpcActor.java:84) ~[flink-rpc-akka_c0c89ca0-3661-4f02-afd5-3d51bafc7d6a.jar:1.16.3]
at org.apache.flink.runtime.rpc.akka.AkkaRpcActor.handleMessage(AkkaRpcActor.java:168) ~[flink-rpc-akka_c0c89ca0-3661-4f02-afd5-3d51bafc7d6a.jar:1.16.3]
at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:24) [flink-rpc-akka_c0c89ca0-3661-4f02-afd5-3d51bafc7d6a.jar:1.16.3]
at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:20) [flink-rpc-akka_c0c89ca0-3661-4f02-afd5-3d51bafc7d6a.jar:1.16.3]
at scala.PartialFunction.applyOrElse(PartialFunction.scala:123) [flink-rpc-akka_c0c89ca0-3661-4f02-afd5-3d51bafc7d6a.jar:1.16.3]
at scala.PartialFunction.applyOrElse$(PartialFunction.scala:122) [flink-rpc-akka_c0c89ca0-3661-4f02-afd5-3d51bafc7d6a.jar:1.16.3]
at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:20) [flink-rpc-akka_c0c89ca0-3661-4f02-afd5-3d51bafc7d6a.jar:1.16.3]
at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:171) [flink-rpc-akka_c0c89ca0-3661-4f02-afd5-3d51bafc7d6a.jar:1.16.3]
at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:172) [flink-rpc-akka_c0c89ca0-3661-4f02-afd5-3d51bafc7d6a.jar:1.16.3]
at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:172) [flink-rpc-akka_c0c89ca0-3661-4f02-afd5-3d51bafc7d6a.jar:1.16.3]
at akka.actor.Actor.aroundReceive(Actor.scala:537) [flink-rpc-akka_c0c89ca0-3661-4f02-afd5-3d51bafc7d6a.jar:1.16.3]
at akka.actor.Actor.aroundReceive$(Actor.scala:535) [flink-rpc-akka_c0c89ca0-3661-4f02-afd5-3d51bafc7d6a.jar:1.16.3]
at akka.actor.AbstractActor.aroundReceive(AbstractActor.scala:220) [flink-rpc-akka_c0c89ca0-3661-4f02-afd5-3d51bafc7d6a.jar:1.16.3]
at akka.actor.ActorCell.receiveMessage(ActorCell.scala:580) [flink-rpc-akka_c0c89ca0-3661-4f02-afd5-3d51bafc7d6a.jar:1.16.3]
at akka.actor.ActorCell.invoke(ActorCell.scala:548) [flink-rpc-akka_c0c89ca0-3661-4f02-afd5-3d51bafc7d6a.jar:1.16.3]
at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:270) [flink-rpc-akka_c0c89ca0-3661-4f02-afd5-3d51bafc7d6a.jar:1.16.3]
at akka.dispatch.Mailbox.run(Mailbox.scala:231) [flink-rpc-akka_c0c89ca0-3661-4f02-afd5-3d51bafc7d6a.jar:1.16.3]
at akka.dispatch.Mailbox.exec(Mailbox.scala:243) [flink-rpc-akka_c0c89ca0-3661-4f02-afd5-3d51bafc7d6a.jar:1.16.3]
at java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:289) [?:1.8.0_141]
at java.util.concurrent.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1056) [?:1.8.0_141]
at java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1692) [?:1.8.0_141]
at java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:157) [?:1.8.0_141]
2024-08-06 17:06:13,523 INFO org.apache.flink.runtime.dispatcher.StandaloneDispatcher [] - Job 5e97c8b468981fcb1d2efcb37c6996a5 has been registered for cleanup in the JobResultStore after reaching a terminal state.
2024-08-06 17:06:13,587 ERROR org.apache.flink.runtime.rest.handler.job.JobStatusHandler [] - Unhandled exception.
org.apache.flink.runtime.messages.FlinkJobNotFoundException: Could not find Flink job (5e97c8b468981fcb1d2efcb37c6996a5)
提取报错关键信息:
Caused by: java.io.InvalidClassException: com.ververica.cdc.connectors.shaded.com.fasterxml.jackson.databind.cfg.MapperConfig; incompatible types for field _mapperFeatures
at java.io.ObjectStreamClass.matchFields(ObjectStreamClass.java:2299)
at java.io.ObjectStreamClass.getReflector(ObjectStreamClass.java:2193)
at java.io.ObjectStreamClass.initNonProxy(ObjectStreamClass.java:669)
at java.io.ObjectInputStream.readNonProxyDesc(ObjectInputStream.java:1843)
at java.io.ObjectInputStream.readClassDesc(ObjectInputStream.java:1713)
at java.io.ObjectInputStream.readNonProxyDesc(ObjectInputStream.java:1843)
at java.io.ObjectInputStream.readClassDesc(ObjectInputStream.java:1713)
at java.io.ObjectInputStream.readNonProxyDesc(ObjectInputStream.java:1843)
at java.io.ObjectInputStream.readClassDesc(ObjectInputStream.java:1713)
at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:2000)
at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1535)
at java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:2245)
at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:2169)
at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:2027)
at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1535)
at java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:2245)
at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:2169)
at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:2027)
at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1535)
at java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:2245)
at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:2169)
at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:2027)
at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1535)
at java.io.ObjectInputStream.readArray(ObjectInputStream.java:1933)
at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1529)
at java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:2245)
at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:2169)
at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:2027)
at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1535)
at java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:2245)
at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:2169)
at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:2027)
at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1535)
at java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:2245)
at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:2169)
at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:2027)
at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1535)
at java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:2245)
at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:2169)
at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:2027)
at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1535)
at java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:2245)
at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:2169)
at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:2027)
at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1535)
at java.io.ObjectInputStream.readObject(ObjectInputStream.java:422)
at org.apache.flink.util.InstantiationUtil.deserializeObject(InstantiationUtil.java:617)
at org.apache.flink.util.InstantiationUtil.deserializeObject(InstantiationUtil.java:602)
at org.apache.flink.util.InstantiationUtil.deserializeObject(InstantiationUtil.java:589)
at org.apache.flink.util.SerializedValue.deserializeValue(SerializedValue.java:67)
at org.apache.flink.runtime.operators.coordination.OperatorCoordinatorHolder.create(OperatorCoordinatorHolder.java:488)
at org.apache.flink.runtime.executiongraph.ExecutionJobVertex.createOperatorCoordinatorHolder(ExecutionJobVertex.java:286)
at org.apache.flink.runtime.executiongraph.ExecutionJobVertex.initialize(ExecutionJobVertex.java:223)
... 20 more
2024-08-06 17:06:13,522 INFO org.apache.flink.runtime.dispatcher.StandaloneDispatcher [] - Could not store completed job Flink Demo - To sink_test(5e97c8b468981fcb1d2efcb37c6996a5).
搜索之后发现,是jackson 版本的问题
按照网上资料调试版本,均没有成功。
最后排除法则确定了可以兼容版本
服务器flink版本: 1.16.3 springBoot flink 版本:1.16.3 flink-cdc 版本:2.3.0 jsckson 版本:2.11.2
报错信息提取自Flink 服务器日志 springboot 项目中只有项目代码运行失败报错
执行成功日志:
17:28:42.256 [http-nio-9040-exec-1] INFO o.a.c.c.C.[.[.[/realTimeCollection] - [log,173] - Initializing Spring DispatcherServlet 'dispatcherServlet'
17:28:53.086 [Flink-RestClusterClient-IO-thread-1] INFO o.a.f.c.p.r.RestClusterClient - [lambda$submitJob$7,401] - Submitting job 'Flink Demo - To sink_test' (501aa403900a830235e14e256a93bf0b).
17:28:55.263 [Flink-RestClusterClient-IO-thread-4] INFO o.a.f.c.p.r.RestClusterClient - [lambda$null$6,420] - Successfully submitted job 'Flink Demo - To sink_test' (501aa403900a830235e14e256a93bf0b) to 'http://10.80.19.142:8081'.
17:28:55.460 [Flink-RestClusterClient-IO-thread-1] INFO o.a.f.c.p.r.RestClusterClient - [lambda$submitJob$7,401] - Submitting job 'Flink Demo - To sink_test_second' (611b2d6ecac103a18e8bfec6bdad0c38).
17:28:55.516 [Flink-RestClusterClient-IO-thread-4] INFO o.a.f.c.p.r.RestClusterClient - [lambda$null$6,420] - Successfully submitted job 'Flink Demo - To sink_test_second' (611b2d6ecac103a18e8bfec6bdad0c38) to 'http://10.80.19.142:8081'.