个人实时计算Flinkcdc学习项目-flinkcdc调试、打包、部署手册

开发调试

调试加入：TableResult transformDml = tableEnv.executeSql(transformDmlSQL);
transformDml.print();

java.lang.ClassNotFoundException: org.apache.kafka.common.utils.ThreadUtils
参考：https://blog.csdn.net/qq_31412425/article/details/131878495
原因：与flink-kafka依赖冲突
解决措施：去掉依赖，重启idea

Caused by: java.lang.IllegalAccessError: tried to access field com.mysql.cj.CharsetMapping.COLLATION
参考：https://blog.csdn.net/qq_27721169/article/details/132151345
原因：版本不匹配
解决措施：降低mysql-connector-java到8.0.26以下

问题：Caused by: java.lang.ClassNotFoundException: org.apache.flink.table.connector.source.lookup.cache.LookupCache
原因：

mysql查看连接数：https://blog.51cto.com/u_16099273/6512586
show variables like "char%";
show global variables like '%time_zone%';
查看mysql版本
SHOW VARIABLES LIKE 'version';

flinkcdc抽取oracle数据：https://blog.csdn.net/ask_baidu/article/details/128510470
flink watermark : https://www.cnblogs.com/jiashengmei/p/15852907.html
env.getConfig().setAutoWatermarkInterval(100); watermark设定固定间隔100

打包

1. 修改pom的scope
2. maven运行clean compile package
3. 打包方式详解：https://blog.csdn.net/m0_55868614/article/details/125329582

部署

1. 进入到flink/bin
2. ./flink run xxx.jar
   ./flink run -c org.example.FlinkCDCdemo1 flink-test-2023112302.jar
   -m：指定提交到的JobManager
   -c：指定入口类

./flink run -c org.example.flinkcdc1 flinkcdc2023112.jar -C flink:/opt/apps/flink-1.13.6/bin/flink-test-1.0-20231123.jar -yt /opt/apps/flink-1.13.6/bin/flink-test-1.0-20231123.jar
./flink run -c org.example.FlinkCDCdemo1 flink-test-1.0-20231123.jar

指定依赖：flink打jar包时只打包代码，依赖jar包批量指定
参考：https://www.cnblogs.com/star521/p/15151571.html

报错参考：https://blog.csdn.net/a123147abc/article/details/119905956

报错：org.apache.flink.client.program.ProgramInvocationException: Neither a 'Main-Class', nor a 'program-class' entry was found in the jar file.
解决：https://blog.csdn.net/weixin_44168526/article/details/123448037

报错：org.apache.flink.client.program.ProgramInvocationException: The program's entry point class 'org.example.FlinkCDCdemo1' was not found in the jar file.
原因：通过bulid方式打包，没有把主程序打入jar包

报错：Caused by: org.apache.flink.table.api.ValidationException: Could not find any factory for identifier 'mysql-cdc' that implements 'org.apache.flink.table.factories.DynamicTableFactory' in the classpath.
原因：jar包没有打入依赖包
解决：加入打入依赖包的插件
参考：（打包方式详解）：https://blog.csdn.net/m0_55868614/article/details/125329582

报错：Caused by: java.lang.SecurityException: Invalid signature file digest for Manifest main attributes
解决：pom加入filters：https://blog.csdn.net/weixin_43253207/article/details/122997458

报错：Caused by: org.apache.flink.table.api.ValidationException: Could not find any factory for identifier 'jdbc' that implements 'org.apache.flink.table.factories.DynamicTableFactory' in the classpath.
解决：flink 运行环境下的lib目录 jdbc jar包出现不同版本的jdbc导致冲突了，删除其中一个即可

报错：org.apache.flink.table.executor.StreamExecutorFactory
原因：暂时认为是程序依赖jar包和环境jar包冲突

报错：Caused by: java.lang.ClassCastException: org.codehaus.janino.CompilerFactory cannot be cast to org.codehaus.commons.compiler.ICompilerFactory