azkaban打包shell脚本，脚本不存在问题
测试azkaban任务执行sh脚本的时候，发现的问题
条件脚本文件都在 condition 文件夹下
如图
但是每次打包上传到azkaban的时候 到执行脚本的时候，都会报脚本不存在。
错误命令
09-01-2023 10:18:20 GMT JobA INFO - Starting job JobA at 1673259500613
09-01-2023 10:18:20 GMT JobA INFO - job JVM args: '-Dazkaban.flowid=condition' '-Dazkaban.execid=46' '-Dazkaban.jobid=JobA'
09-01-2023 10:18:20 GMT JobA INFO - user.to.proxy property was not set, defaulting to submit user iullor
09-01-2023 10:18:20 GMT JobA INFO - Attached Ramp Props : [{}]
09-01-2023 10:18:20 GMT JobA INFO - Building command job executor.
09-01-2023 10:18:20 GMT JobA INFO - Failed with 5 inputs with exception e = null
09-01-2023 10:18:20 GMT JobA INFO - Memory granted for job JobA
09-01-2023 10:18:20 GMT JobA INFO - 1 commands to execute.
09-01-2023 10:18:20 GMT JobA INFO - cwd=/home/azkaban/azkaban-exec-server-3.84.4/executions/46
09-01-2023 10:18:20 GMT JobA INFO - effective user is: iullor
09-01-2023 10:18:20 GMT JobA INFO - Command: sh ../JobA.sh
09-01-2023 10:18:20 GMT JobA INFO - Environment variables: {JOB_OUTPUT_PROP_FILE=/home/azkaban/azkaban-exec-server-3.84.4/executions/46/JobA_output_6778758939652178803_tmp, JOB_PROP_FILE=/home/azkaban/azkaban-exec-server-3.84.4/executions/46/JobA_job_props_2360083906763032481_tmp, KRB5CCNAME=/tmp/krb5cc__condition__condition__JobA__46__iullor, JOB_NAME=JobA}
09-01-2023 10:18:20 GMT JobA INFO - Working directory: /home/azkaban/azkaban-exec-server-3.84.4/executions/46
09-01-2023 10:18:20 GMT JobA INFO - Spawned process with id 5023
09-01-2023 10:18:20 GMT JobA INFO - sh: ../JobA.sh: No such file or directory
09-01-2023 10:18:20 GMT JobA INFO - Process with id 5023 completed unsuccessfully in 0 seconds.
09-01-2023 10:18:20 GMT JobA ERROR - Job run failed!
java.lang.RuntimeException: azkaban.jobExecutor.utils.process.ProcessFailureException: Process exited with code 127
at azkaban.jobExecutor.ProcessJob.run(ProcessJob.java:312)
at azkaban.execapp.JobRunner.runJob(JobRunner.java:823)
at azkaban.execapp.JobRunner.doRun(JobRunner.java:602)
at azkaban.execapp.JobRunner.run(JobRunner.java:563)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Caused by: azkaban.jobExecutor.utils.process.ProcessFailureException: Process exited with code 127
at azkaban.jobExecutor.utils.process.AzkabanProcess.run(AzkabanProcess.java:125)
at azkaban.jobExecutor.ProcessJob.run(ProcessJob.java:304)
... 8 more
09-01-2023 10:18:20 GMT JobA ERROR - azkaban.jobExecutor.utils.process.ProcessFailureException: Process exited with code 127 cause: azkaban.jobExecutor.utils.process.ProcessFailureException: Process exited with code 127
09-01-2023 10:18:20 GMT JobA INFO - Finishing job JobA at 1673259500990 with status FAILED
三个原因导致的
文件编码格式
- 脚本格式是 windows UTF8 的 ，Linux系统不认
修改方式，使用notpad++ 将文件格式转换成linux的 如图
job包打包方式
- 打包目录位置问题，如果用360打包，如果在文件的外层打包，默认打包后的位置，会多一层，也就是zip包点进去的时候多了一层目录，博主就是被这个问题搞了一天，奶的，azkaban真垃圾
错误
正确
某台服务器上的脚本 ，在多个executor集群执行时
- 如果脚本文件放在某台服务器上，但是azkaban却配置的集群，那么如果执行任务被分配到了不同的机器上，按照绝对路径找，则文件不存在
解决方式1:在执行的时候，指定任务在某台机器上执行，给任务 配置某台executor的ID
解决方式2:使用分发命令，将可执行的脚本文件分发到各个集群环境
不说了，我要去研究dolphinscheduler啦
