Azkaban 系列文章
1、azkaban介绍、三种（solo-server、two-server和multiple-executor）部署方式及验证 2、azkaban-3.51.0 任务类型（Jobtypes，详细介绍command、java2种类型及示例） 3、azkaban-3.51.0 条件工作流flow和参数传递 4、azkaban-3.51.0 通过api进行操作azkaban界面功能
本文的前提是上一篇的azkaban使用介绍，因为本文的示例不会再说怎么上传、运行，而是直接给出源文件与运行结果。 本文分为四个部分，即azkaban支持的类型、command类型示例、java类型示例和任务调度。
一、Azkaban支持的plugin类型
- command：Linux shell命令行任务
- gobblin：通用数据采集工具
- hadoopJava：运行hadoopMR任务
- java：原生java任务
- hive：支持执行hiveSQL
- pig：pig脚本任务
- spark：spark任务
- hdfsToTeradata：把数据从hdfs导入Teradata
- teradataToHdfs：把数据从Teradata导入hdfs
二、command类型
1、 job配置（示例）示例一
- test.job
type=command
command=echo "This is azkaban cmd ... "
command.1=whoami
#依赖前一个job
dependencies=cmd1
- cmd1.job
type=command
command=echo "This is azkaban cmd1 ... "
- 运行日志 test.job
18-08-2022 11:10:12 CST cmd1 INFO - Starting job cmd1 at 1660792212782
18-08-2022 11:10:12 CST cmd1 INFO - job JVM args: -Dazkaban.flowid=test -Dazkaban.execid=36 -Dazkaban.jobid=cmd1
18-08-2022 11:10:12 CST cmd1 INFO - user.to.proxy property was not set, defaulting to submit user azkaban
18-08-2022 11:10:12 CST cmd1 INFO - Building command job executor.
18-08-2022 11:10:12 CST cmd1 INFO - Memory granted for job cmd1
18-08-2022 11:10:12 CST cmd1 INFO - 1 commands to execute.
18-08-2022 11:10:12 CST cmd1 INFO - cwd=/usr/local/bigdata/azkaban3.51.0/exec-server/azkaban-exec-server-0.1.0-SNAPSHOT/bin/executions/36
18-08-2022 11:10:12 CST cmd1 INFO - effective user is: azkaban
18-08-2022 11:10:12 CST cmd1 INFO - Command: echo "This is azkaban cmd1 ... "
18-08-2022 11:10:12 CST cmd1 INFO - Environment variables: {JOB_OUTPUT_PROP_FILE=/usr/local/bigdata/azkaban3.51.0/exec-server/azkaban-exec-server-0.1.0-SNAPSHOT/bin/executions/36/cmd1_output_7156127205290501411_tmp, JOB_PROP_FILE=/usr/local/bigdata/azkaban3.51.0/exec-server/azkaban-exec-server-0.1.0-SNAPSHOT/bin/executions/36/cmd1_props_4986904569339043450_tmp, KRB5CCNAME=/tmp/krb5cc__mutilexec__test__cmd1__36__azkaban, JOB_NAME=cmd1}
18-08-2022 11:10:12 CST cmd1 INFO - Working directory: /usr/local/bigdata/azkaban3.51.0/exec-server/azkaban-exec-server-0.1.0-SNAPSHOT/bin/executions/36
18-08-2022 11:10:12 CST cmd1 INFO - This is azkaban cmd1 ...
18-08-2022 11:10:12 CST cmd1 INFO - Process completed successfully in 0 seconds.
18-08-2022 11:10:12 CST cmd1 INFO - output properties file=/usr/local/bigdata/azkaban3.51.0/exec-server/azkaban-exec-server-0.1.0-SNAPSHOT/bin/executions/36/cmd1_output_7156127205290501411_tmp
18-08-2022 11:10:12 CST cmd1 INFO - Finishing job cmd1 at 1660792212797 with status SUCCEEDED
cmd1.job
18-08-2022 11:10:12 CST cmd1 INFO - Starting job cmd1 at 1660792212782
18-08-2022 11:10:12 CST cmd1 INFO - job JVM args: -Dazkaban.flowid=test -Dazkaban.execid=36 -Dazkaban.jobid=cmd1
18-08-2022 11:10:12 CST cmd1 INFO - user.to.proxy property was not set, defaulting to submit user azkaban
18-08-2022 11:10:12 CST cmd1 INFO - Building command job executor.
18-08-2022 11:10:12 CST cmd1 INFO - Memory granted for job cmd1
18-08-2022 11:10:12 CST cmd1 INFO - 1 commands to execute.
18-08-2022 11:10:12 CST cmd1 INFO - cwd=/usr/local/bigdata/azkaban3.51.0/exec-server/azkaban-exec-server-0.1.0-SNAPSHOT/bin/executions/36
18-08-2022 11:10:12 CST cmd1 INFO - effective user is: azkaban
18-08-2022 11:10:12 CST cmd1 INFO - Command: echo "This is azkaban cmd1 ... "
18-08-2022 11:10:12 CST cmd1 INFO - Environment variables: {JOB_OUTPUT_PROP_FILE=/usr/local/bigdata/azkaban3.51.0/exec-server/azkaban-exec-server-0.1.0-SNAPSHOT/bin/executions/36/cmd1_output_7156127205290501411_tmp, JOB_PROP_FILE=/usr/local/bigdata/azkaban3.51.0/exec-server/azkaban-exec-server-0.1.0-SNAPSHOT/bin/executions/36/cmd1_props_4986904569339043450_tmp, KRB5CCNAME=/tmp/krb5cc__mutilexec__test__cmd1__36__azkaban, JOB_NAME=cmd1}
18-08-2022 11:10:12 CST cmd1 INFO - Working directory: /usr/local/bigdata/azkaban3.51.0/exec-server/azkaban-exec-server-0.1.0-SNAPSHOT/bin/executions/36
18-08-2022 11:10:12 CST cmd1 INFO - This is azkaban cmd1 ...
18-08-2022 11:10:12 CST cmd1 INFO - Process completed successfully in 0 seconds.
18-08-2022 11:10:12 CST cmd1 INFO - output properties file=/usr/local/bigdata/azkaban3.51.0/exec-server/azkaban-exec-server-0.1.0-SNAPSHOT/bin/executions/36/cmd1_output_7156127205290501411_tmp
18-08-2022 11:10:12 CST cmd1 INFO - Finishing job cmd1 at 1660792212797 with status SUCCEEDED
2、 job配置（依赖）示例二
- basic.flow
---
config:
failure.emails: noreply@foo.com
nodes:
- name: jobC
type: noop
# jobC depends on jobA and jobB
dependsOn:
- jobA
- jobB
- name: jobA
type: command
config:
command: echo "This is an echoed text."
- name: jobB
type: command
config:
command: pwd
- 图形依赖示例
- 执行结果
- 运行日志 JobB
18-08-2022 13:51:03 CST jobB INFO - Starting job jobB at 1660801863329
18-08-2022 13:51:03 CST jobB INFO - job JVM args: -Dazkaban.flowid=basic -Dazkaban.execid=37 -Dazkaban.jobid=jobB
18-08-2022 13:51:03 CST jobB INFO - user.to.proxy property was not set, defaulting to submit user azkaban
18-08-2022 13:51:03 CST jobB INFO - Building command job executor.
18-08-2022 13:51:03 CST jobB INFO - Memory granted for job jobB
18-08-2022 13:51:03 CST jobB INFO - 1 commands to execute.
18-08-2022 13:51:03 CST jobB INFO - cwd=/usr/local/bigdata/azkaban3.51.0/exec-server/azkaban-exec-server-0.1.0-SNAPSHOT/bin/executions/37
18-08-2022 13:51:03 CST jobB INFO - effective user is: azkaban
18-08-2022 13:51:03 CST jobB INFO - Command: pwd
18-08-2022 13:51:03 CST jobB INFO - Environment variables: {JOB_OUTPUT_PROP_FILE=/usr/local/bigdata/azkaban3.51.0/exec-server/azkaban-exec-server-0.1.0-SNAPSHOT/bin/executions/37/jobB_output_3128039260514846300_tmp, JOB_PROP_FILE=/usr/local/bigdata/azkaban3.51.0/exec-server/azkaban-exec-server-0.1.0-SNAPSHOT/bin/executions/37/jobB_props_7802985401895047938_tmp, KRB5CCNAME=/tmp/krb5cc__mutilexec__basic__jobB__37__azkaban, JOB_NAME=jobB}
18-08-2022 13:51:03 CST jobB INFO - Working directory: /usr/local/bigdata/azkaban3.51.0/exec-server/azkaban-exec-server-0.1.0-SNAPSHOT/bin/executions/37
18-08-2022 13:51:03 CST jobB INFO - /usr/local/bigdata/azkaban3.51.0/exec-server/azkaban-exec-server-0.1.0-SNAPSHOT/bin/executions/37
18-08-2022 13:51:03 CST jobB INFO - Process completed successfully in 0 seconds.
18-08-2022 13:51:03 CST jobB INFO - output properties file=/usr/local/bigdata/azkaban3.51.0/exec-server/azkaban-exec-server-0.1.0-SNAPSHOT/bin/executions/37/jobB_output_3128039260514846300_tmp
18-08-2022 13:51:03 CST jobB INFO - Finishing job jobB at 1660801863361 with status SUCCEEDED
JobA
18-08-2022 13:51:03 CST jobA INFO - Starting job jobA at 1660801863336
18-08-2022 13:51:03 CST jobA INFO - job JVM args: -Dazkaban.flowid=basic -Dazkaban.execid=37 -Dazkaban.jobid=jobA
18-08-2022 13:51:03 CST jobA INFO - user.to.proxy property was not set, defaulting to submit user azkaban
18-08-2022 13:51:03 CST jobA INFO - Building command job executor.
18-08-2022 13:51:03 CST jobA INFO - Memory granted for job jobA
18-08-2022 13:51:03 CST jobA INFO - 1 commands to execute.
18-08-2022 13:51:03 CST jobA INFO - cwd=/usr/local/bigdata/azkaban3.51.0/exec-server/azkaban-exec-server-0.1.0-SNAPSHOT/bin/executions/37
18-08-2022 13:51:03 CST jobA INFO - effective user is: azkaban
18-08-2022 13:51:03 CST jobA INFO - Command: echo "This is an echoed text."
18-08-2022 13:51:03 CST jobA INFO - Environment variables: {JOB_OUTPUT_PROP_FILE=/usr/local/bigdata/azkaban3.51.0/exec-server/azkaban-exec-server-0.1.0-SNAPSHOT/bin/executions/37/jobA_output_1861938410828184844_tmp, JOB_PROP_FILE=/usr/local/bigdata/azkaban3.51.0/exec-server/azkaban-exec-server-0.1.0-SNAPSHOT/bin/executions/37/jobA_props_7973309314170642567_tmp, KRB5CCNAME=/tmp/krb5cc__mutilexec__basic__jobA__37__azkaban, JOB_NAME=jobA}
18-08-2022 13:51:03 CST jobA INFO - Working directory: /usr/local/bigdata/azkaban3.51.0/exec-server/azkaban-exec-server-0.1.0-SNAPSHOT/bin/executions/37
18-08-2022 13:51:03 CST jobA INFO - This is an echoed text.
18-08-2022 13:51:03 CST jobA INFO - Process completed successfully in 0 seconds.
18-08-2022 13:51:03 CST jobA INFO - output properties file=/usr/local/bigdata/azkaban3.51.0/exec-server/azkaban-exec-server-0.1.0-SNAPSHOT/bin/executions/37/jobA_output_1861938410828184844_tmp
18-08-2022 13:51:03 CST jobA INFO - Finishing job jobA at 1660801863361 with status SUCCEEDED
JobC
18-08-2022 13:51:03 CST jobC INFO - Starting job jobC at 1660801863373
18-08-2022 13:51:03 CST jobC INFO - job JVM args: -Dazkaban.flowid=basic -Dazkaban.execid=37 -Dazkaban.jobid=jobC
18-08-2022 13:51:03 CST jobC INFO - user.to.proxy property was not set, defaulting to submit user azkaban
18-08-2022 13:51:03 CST jobC INFO - Building noop job executor.
18-08-2022 13:51:03 CST jobC INFO - Finishing job jobC at 1660801863383 with status SUCCEEDED
3、 job配置（嵌入式）示例三
- embedded.flow
---
config:
failure.emails: noreply@foo.com
nodes:
- name: jobC
type: noop
dependsOn:
- embedded_flow
- name: embedded_flow
type: flow
config:
prop: value
nodes:
- name: jobB
type: noop
dependsOn:
- jobA
- name: jobA
type: command
config:
command: pwd
- 图示
- 运行结果
- 运行日志
18-08-2022 13:59:03 CST jobC INFO - Starting job jobC at 1660802343666
18-08-2022 13:59:03 CST jobC INFO - job JVM args: -Dazkaban.flowid=embedded -Dazkaban.execid=38 -Dazkaban.jobid=jobC
18-08-2022 13:59:03 CST jobC INFO - user.to.proxy property was not set, defaulting to submit user azkaban
18-08-2022 13:59:03 CST jobC INFO - Building noop job executor.
18-08-2022 13:59:03 CST jobC INFO - Finishing job jobC at 1660802343677 with status SUCCEEDED
4、job配置示（hdfs）例四
本示例是hdfs创建目录
- 创建job
# fs.job
type=command
command=/usr/local/bigdata/hadoop-3.1.4/bin/hadoop fs -mkdir /azkatest
- 将job资源文件打包成zip文件
- 通过azkaban的web管理平台创建project并上传job压缩包
- 启动执行该job
5、job配置（mapreduce）示例五
执行mapreduce任务，执行示例为wordcount。
- 创建job
# mrwc.job
type=command
command=/usr/local/bigdata/hadoop-3.1.4/bin/hadoop jar hadoop-mapreduce-examples-2.6.1.jar wordcount /wordcount/input /wordcount/azout
- 将job资源文件打包成zip文件
- 通过azkaban的web管理平台创建project并上传job压缩包
- 启动执行该job
6、job配置（hive）示例六
执行hive的sql
- Hive脚本， test.sql 假设数据文件都是准备好的，hadoop、hive环境都是好用的
use default;
drop table aztest;
create table aztest(id int,name string) row format delimited fields terminated by ',';
load data inpath '/aztest/hiveinput' into table aztest;
create table azres as select * from aztest;
insert overwrite directory '/aztest/hiveoutput' select count(1) from aztest;
- 创建job
# hivef.job
type=command
command=/usr/local/bigdata/hive/bin/hive -f 'test.sql'
- 将job资源文件打包成zip文件
- 通过azkaban的web管理平台创建project并上传job压缩包
- 启动执行该job
三、java类型
1、 job配置（普通java）示例一
1）、创建job
- testjavajob.job
type=javaprocess
classpath=./lib/*,${azkaban.home}/lib/*
java.class=com.win.TestAzkabanType param1 param2
2）、TestAzkabanType.java
package com.win;
public class TestAzkabanType {
public static void main(String[] args) {
System.out.println("this is a java class. java type,package:com.win! 参数1="+args[0]+" 参数2= "+args[1]);
}
}
3）、打包
将testjavajob.job和TestAzkabanType.jar打包成一个zip文件，并上传至azkaban中
4）、运行
5）、运行日志
22-08-2022 16:32:51 CST testjavajob INFO - Starting job testjavajob at 1661157171968
22-08-2022 16:32:51 CST testjavajob INFO - job JVM args: -Dazkaban.flowid=testjavajob -Dazkaban.execid=94 -Dazkaban.jobid=testjavajob
22-08-2022 16:32:51 CST testjavajob INFO - user.to.proxy property was not set, defaulting to submit user azkaban
22-08-2022 16:32:51 CST testjavajob INFO - Building javaprocess job executor.
22-08-2022 16:32:51 CST testjavajob INFO - Memory granted for job testjavajob
22-08-2022 16:32:51 CST testjavajob INFO - No classpath specified. Trying to load classes from /usr/local/bigdata/azkaban3.51.0/exec-server/azkaban-exec-server-0.1.0-SNAPSHOT/bin/executions/94
22-08-2022 16:32:51 CST testjavajob INFO - 1 commands to execute.
22-08-2022 16:32:51 CST testjavajob INFO - cwd=/usr/local/bigdata/azkaban3.51.0/exec-server/azkaban-exec-server-0.1.0-SNAPSHOT/bin/executions/94
22-08-2022 16:32:51 CST testjavajob INFO - effective user is: azkaban
22-08-2022 16:32:51 CST testjavajob INFO - Command: java -Dazkaban.flowid=testjavajob -Dazkaban.execid=94 -Dazkaban.jobid=testjavajob -Xms64M -Xmx256M -cp TestAzkabanType.jar com.win.TestAzkabanType param1 param2
22-08-2022 16:32:51 CST testjavajob INFO - Environment variables: {JOB_OUTPUT_PROP_FILE=/usr/local/bigdata/azkaban3.51.0/exec-server/azkaban-exec-server-0.1.0-SNAPSHOT/bin/executions/94/testjavajob_output_7286382865992072360_tmp, JOB_PROP_FILE=/usr/local/bigdata/azkaban3.51.0/exec-server/azkaban-exec-server-0.1.0-SNAPSHOT/bin/executions/94/testjavajob_props_8373462308281870001_tmp, KRB5CCNAME=/tmp/krb5cc__mutilexec__testjavajob__testjavajob__94__azkaban, JOB_NAME=testjavajob}
22-08-2022 16:32:51 CST testjavajob INFO - Working directory: /usr/local/bigdata/azkaban3.51.0/exec-server/azkaban-exec-server-0.1.0-SNAPSHOT/bin/executions/94
22-08-2022 16:32:52 CST testjavajob INFO - this is a java class. java type,package:com.win! 参数1=param1 参数2= param2
22-08-2022 16:32:52 CST testjavajob INFO - Process completed successfully in 0 seconds.
22-08-2022 16:32:52 CST testjavajob INFO - output properties file=/usr/local/bigdata/azkaban3.51.0/exec-server/azkaban-exec-server-0.1.0-SNAPSHOT/bin/executions/94/testjavajob_output_7286382865992072360_tmp
22-08-2022 16:32:52 CST testjavajob INFO - Finishing job testjavajob at 1661157172037 with status SUCCEEDED
其日志中有执行命令，实际情况可以根据该命令确定执行的main-class，看执行命令好像打包的时候不需要指定具体的main-class
22-08-2022 16:32:51 CST testjavajob INFO - Command: java -Dazkaban.flowid=testjavajob -Dazkaban.execid=94 -Dazkaban.jobid=testjavajob -Xms64M -Xmx256M -cp TestAzkabanType.jar com.win.TestAzkabanType param1 param2
如果涉及到引用jdk外的包，需要一并打包进jar即可（或指定具体的目录结构，形如：classpath=./lib/*,${azkaban.home}/lib/*）
2、job配置（依赖）示例三
1）、创建java文件并打包成jar
参见示例1
2）、创建job
- innerFlow.job
type=javaprocess
java.class=com.win.TestAzkabanType innerFlow dependencies=innerJobB,innerJobC
seconds=1
fail=false
dependencies=innerJobB,innerJobC
- innerJobC.job
type=javaprocess
java.class=com.win.TestAzkabanType innerJobC innerJobA
seconds=1
fail=false
dependencies=innerJobA
- innerJobA.job
type=javaprocess
java.class=com.win.TestAzkabanType innerJobA nothing
seconds=1
fail=false
- innerJobB.job
type=javaprocess
java.class=com.win.TestAzkabanType innerJobB innerJobA
seconds=1
fail=false
dependencies=innerJobA
3）、打包
4个job和一个jar文件一起打包，并上传即可
4）、运行
5）、运行日志
- 整体运行日志
22-08-2022 16:44:59 CST innerFlow INFO - Assigned executor : localhost:12321
22-08-2022 16:44:59 CST innerFlow INFO - Running execid:95 flow:innerFlow project:6 version:56
22-08-2022 16:44:59 CST innerFlow INFO - Updating initial flow directory.
22-08-2022 16:44:59 CST innerFlow INFO - Fetching job and shared properties.
22-08-2022 16:44:59 CST innerFlow INFO - Starting flows
22-08-2022 16:44:59 CST innerFlow INFO - Running flow 'innerFlow'.
22-08-2022 16:44:59 CST innerFlow INFO - Configuring Azkaban metrics tracking for jobrunner object
22-08-2022 16:44:59 CST innerFlow INFO - Submitting job 'innerJobA' to run.
22-08-2022 16:44:59 CST innerFlow INFO - Created file appender for job innerJobA
22-08-2022 16:44:59 CST innerFlow INFO - Attached file appender for job innerJobA
22-08-2022 16:44:59 CST innerFlow INFO - Job Started: innerJobA
22-08-2022 16:45:00 CST innerFlow INFO - No attachment file for job innerJobA written.
22-08-2022 16:45:00 CST innerFlow INFO - Job innerJobA finished with status SUCCEEDED in 0 seconds
22-08-2022 16:45:00 CST innerFlow INFO - Configuring Azkaban metrics tracking for jobrunner object
22-08-2022 16:45:00 CST innerFlow INFO - Submitting job 'innerJobB' to run.
22-08-2022 16:45:00 CST innerFlow INFO - Created file appender for job innerJobB
22-08-2022 16:45:00 CST innerFlow INFO - Attached file appender for job innerJobB
22-08-2022 16:45:00 CST innerFlow INFO - Configuring Azkaban metrics tracking for jobrunner object
22-08-2022 16:45:00 CST innerFlow INFO - Submitting job 'innerJobC' to run.
22-08-2022 16:45:00 CST innerFlow INFO - Job Started: innerJobB
22-08-2022 16:45:00 CST innerFlow INFO - Created file appender for job innerJobC
22-08-2022 16:45:00 CST innerFlow INFO - Attached file appender for job innerJobC
22-08-2022 16:45:00 CST innerFlow INFO - Job Started: innerJobC
22-08-2022 16:45:00 CST innerFlow INFO - No attachment file for job innerJobB written.
22-08-2022 16:45:00 CST innerFlow INFO - Job innerJobB finished with status SUCCEEDED in 0 seconds
22-08-2022 16:45:00 CST innerFlow INFO - No attachment file for job innerJobC written.
22-08-2022 16:45:00 CST innerFlow INFO - Job innerJobC finished with status SUCCEEDED in 0 seconds
22-08-2022 16:45:00 CST innerFlow INFO - Configuring Azkaban metrics tracking for jobrunner object
22-08-2022 16:45:00 CST innerFlow INFO - Submitting job 'innerFlow' to run.
22-08-2022 16:45:00 CST innerFlow INFO - Created file appender for job innerFlow
22-08-2022 16:45:00 CST innerFlow INFO - Attached file appender for job innerFlow
22-08-2022 16:45:00 CST innerFlow INFO - Job Started: innerFlow
22-08-2022 16:45:00 CST innerFlow INFO - No attachment file for job innerFlow written.
22-08-2022 16:45:00 CST innerFlow INFO - Job innerFlow finished with status SUCCEEDED in 0 seconds
22-08-2022 16:45:00 CST innerFlow INFO - Flow '' is set to SUCCEEDED in 0 seconds
22-08-2022 16:45:00 CST innerFlow INFO - Finishing up flow. Awaiting Termination
22-08-2022 16:45:00 CST innerFlow INFO - Finished Flow
22-08-2022 16:45:00 CST innerFlow INFO - Setting end time for flow 95 to 1661157900207
- innerFlow.job
22-08-2022 16:45:00 CST innerFlow INFO - Starting job innerFlow at 1661157900121
22-08-2022 16:45:00 CST innerFlow INFO - job JVM args: -Dazkaban.flowid=innerFlow -Dazkaban.execid=95 -Dazkaban.jobid=innerFlow
22-08-2022 16:45:00 CST innerFlow INFO - user.to.proxy property was not set, defaulting to submit user azkaban
22-08-2022 16:45:00 CST innerFlow INFO - Building javaprocess job executor.
22-08-2022 16:45:00 CST innerFlow INFO - Memory granted for job innerFlow
22-08-2022 16:45:00 CST innerFlow INFO - No classpath specified. Trying to load classes from /usr/local/bigdata/azkaban3.51.0/exec-server/azkaban-exec-server-0.1.0-SNAPSHOT/bin/executions/95
22-08-2022 16:45:00 CST innerFlow INFO - 1 commands to execute.
22-08-2022 16:45:00 CST innerFlow INFO - cwd=/usr/local/bigdata/azkaban3.51.0/exec-server/azkaban-exec-server-0.1.0-SNAPSHOT/bin/executions/95
22-08-2022 16:45:00 CST innerFlow INFO - effective user is: azkaban
22-08-2022 16:45:00 CST innerFlow INFO - Command: java -Dazkaban.flowid=innerFlow -Dazkaban.execid=95 -Dazkaban.jobid=innerFlow -Xms64M -Xmx256M -cp TestAzkabanType.jar com.win.TestAzkabanType innerFlow dependencies=innerJobB,innerJobC
22-08-2022 16:45:00 CST innerFlow INFO - Environment variables: {JOB_OUTPUT_PROP_FILE=/usr/local/bigdata/azkaban3.51.0/exec-server/azkaban-exec-server-0.1.0-SNAPSHOT/bin/executions/95/innerFlow_output_7020976528385638433_tmp, JOB_PROP_FILE=/usr/local/bigdata/azkaban3.51.0/exec-server/azkaban-exec-server-0.1.0-SNAPSHOT/bin/executions/95/innerFlow_props_2379673642283223464_tmp, KRB5CCNAME=/tmp/krb5cc__mutilexec__innerFlow__innerFlow__95__azkaban, JOB_NAME=innerFlow}
22-08-2022 16:45:00 CST innerFlow INFO - Working directory: /usr/local/bigdata/azkaban3.51.0/exec-server/azkaban-exec-server-0.1.0-SNAPSHOT/bin/executions/95
22-08-2022 16:45:00 CST innerFlow INFO - this is a java class. java type,package:com.win! 参数1=innerFlow 参数2= dependencies=innerJobB,innerJobC
22-08-2022 16:45:00 CST innerFlow INFO - Process completed successfully in 0 seconds.
22-08-2022 16:45:00 CST innerFlow INFO - output properties file=/usr/local/bigdata/azkaban3.51.0/exec-server/azkaban-exec-server-0.1.0-SNAPSHOT/bin/executions/95/innerFlow_output_7020976528385638433_tmp
22-08-2022 16:45:00 CST innerFlow INFO - Finishing job innerFlow at 1661157900196 with status SUCCEEDED
- innerJobA.job
22-08-2022 16:44:59 CST innerJobA INFO - Starting job innerJobA at 1661157899947
22-08-2022 16:44:59 CST innerJobA INFO - job JVM args: -Dazkaban.flowid=innerFlow -Dazkaban.execid=95 -Dazkaban.jobid=innerJobA
22-08-2022 16:44:59 CST innerJobA INFO - user.to.proxy property was not set, defaulting to submit user azkaban
22-08-2022 16:44:59 CST innerJobA INFO - Building javaprocess job executor.
22-08-2022 16:44:59 CST innerJobA INFO - Memory granted for job innerJobA
22-08-2022 16:44:59 CST innerJobA INFO - No classpath specified. Trying to load classes from /usr/local/bigdata/azkaban3.51.0/exec-server/azkaban-exec-server-0.1.0-SNAPSHOT/bin/executions/95
22-08-2022 16:44:59 CST innerJobA INFO - 1 commands to execute.
22-08-2022 16:44:59 CST innerJobA INFO - cwd=/usr/local/bigdata/azkaban3.51.0/exec-server/azkaban-exec-server-0.1.0-SNAPSHOT/bin/executions/95
22-08-2022 16:44:59 CST innerJobA INFO - effective user is: azkaban
22-08-2022 16:44:59 CST innerJobA INFO - Command: java -Dazkaban.flowid=innerFlow -Dazkaban.execid=95 -Dazkaban.jobid=innerJobA -Xms64M -Xmx256M -cp TestAzkabanType.jar com.win.TestAzkabanType innerJobA nothing
22-08-2022 16:44:59 CST innerJobA INFO - Environment variables: {JOB_OUTPUT_PROP_FILE=/usr/local/bigdata/azkaban3.51.0/exec-server/azkaban-exec-server-0.1.0-SNAPSHOT/bin/executions/95/innerJobA_output_2048172206987154574_tmp, JOB_PROP_FILE=/usr/local/bigdata/azkaban3.51.0/exec-server/azkaban-exec-server-0.1.0-SNAPSHOT/bin/executions/95/innerJobA_props_8361574289772420710_tmp, KRB5CCNAME=/tmp/krb5cc__mutilexec__innerFlow__innerJobA__95__azkaban, JOB_NAME=innerJobA}
22-08-2022 16:44:59 CST innerJobA INFO - Working directory: /usr/local/bigdata/azkaban3.51.0/exec-server/azkaban-exec-server-0.1.0-SNAPSHOT/bin/executions/95
22-08-2022 16:45:00 CST innerJobA INFO - this is a java class. java type,package:com.win! 参数1=innerJobA 参数2= nothing
22-08-2022 16:45:00 CST innerJobA INFO - Process completed successfully in 0 seconds.
22-08-2022 16:45:00 CST innerJobA INFO - output properties file=/usr/local/bigdata/azkaban3.51.0/exec-server/azkaban-exec-server-0.1.0-SNAPSHOT/bin/executions/95/innerJobA_output_2048172206987154574_tmp
22-08-2022 16:45:00 CST innerJobA INFO - Finishing job innerJobA at 1661157900021 with status SUCCEEDED
- innerJobB.job
22-08-2022 16:45:00 CST innerJobB INFO - Starting job innerJobB at 1661157900029
22-08-2022 16:45:00 CST innerJobB INFO - job JVM args: -Dazkaban.flowid=innerFlow -Dazkaban.execid=95 -Dazkaban.jobid=innerJobB
22-08-2022 16:45:00 CST innerJobB INFO - user.to.proxy property was not set, defaulting to submit user azkaban
22-08-2022 16:45:00 CST innerJobB INFO - Building javaprocess job executor.
22-08-2022 16:45:00 CST innerJobB INFO - Memory granted for job innerJobB
22-08-2022 16:45:00 CST innerJobB INFO - No classpath specified. Trying to load classes from /usr/local/bigdata/azkaban3.51.0/exec-server/azkaban-exec-server-0.1.0-SNAPSHOT/bin/executions/95
22-08-2022 16:45:00 CST innerJobB INFO - 1 commands to execute.
22-08-2022 16:45:00 CST innerJobB INFO - cwd=/usr/local/bigdata/azkaban3.51.0/exec-server/azkaban-exec-server-0.1.0-SNAPSHOT/bin/executions/95
22-08-2022 16:45:00 CST innerJobB INFO - effective user is: azkaban
22-08-2022 16:45:00 CST innerJobB INFO - Command: java -Dazkaban.flowid=innerFlow -Dazkaban.execid=95 -Dazkaban.jobid=innerJobB -Xms64M -Xmx256M -cp TestAzkabanType.jar com.win.TestAzkabanType innerJobB innerJobA
22-08-2022 16:45:00 CST innerJobB INFO - Environment variables: {JOB_OUTPUT_PROP_FILE=/usr/local/bigdata/azkaban3.51.0/exec-server/azkaban-exec-server-0.1.0-SNAPSHOT/bin/executions/95/innerJobB_output_1437460109394890875_tmp, JOB_PROP_FILE=/usr/local/bigdata/azkaban3.51.0/exec-server/azkaban-exec-server-0.1.0-SNAPSHOT/bin/executions/95/innerJobB_props_6395251793150318646_tmp, KRB5CCNAME=/tmp/krb5cc__mutilexec__innerFlow__innerJobB__95__azkaban, JOB_NAME=innerJobB}
22-08-2022 16:45:00 CST innerJobB INFO - Working directory: /usr/local/bigdata/azkaban3.51.0/exec-server/azkaban-exec-server-0.1.0-SNAPSHOT/bin/executions/95
22-08-2022 16:45:00 CST innerJobB INFO - this is a java class. java type,package:com.win! 参数1=innerJobB 参数2= innerJobA
22-08-2022 16:45:00 CST innerJobB INFO - Process completed successfully in 0 seconds.
22-08-2022 16:45:00 CST innerJobB INFO - output properties file=/usr/local/bigdata/azkaban3.51.0/exec-server/azkaban-exec-server-0.1.0-SNAPSHOT/bin/executions/95/innerJobB_output_1437460109394890875_tmp
22-08-2022 16:45:00 CST innerJobB INFO - Finishing job innerJobB at 1661157900102 with status SUCCEEDED
- innerJobC.job
22-08-2022 16:45:00 CST innerJobC INFO - Starting job innerJobC at 1661157900033
22-08-2022 16:45:00 CST innerJobC INFO - job JVM args: -Dazkaban.flowid=innerFlow -Dazkaban.execid=95 -Dazkaban.jobid=innerJobC
22-08-2022 16:45:00 CST innerJobC INFO - user.to.proxy property was not set, defaulting to submit user azkaban
22-08-2022 16:45:00 CST innerJobC INFO - Building javaprocess job executor.
22-08-2022 16:45:00 CST innerJobC INFO - Memory granted for job innerJobC
22-08-2022 16:45:00 CST innerJobC INFO - No classpath specified. Trying to load classes from /usr/local/bigdata/azkaban3.51.0/exec-server/azkaban-exec-server-0.1.0-SNAPSHOT/bin/executions/95
22-08-2022 16:45:00 CST innerJobC INFO - 1 commands to execute.
22-08-2022 16:45:00 CST innerJobC INFO - cwd=/usr/local/bigdata/azkaban3.51.0/exec-server/azkaban-exec-server-0.1.0-SNAPSHOT/bin/executions/95
22-08-2022 16:45:00 CST innerJobC INFO - effective user is: azkaban
22-08-2022 16:45:00 CST innerJobC INFO - Command: java -Dazkaban.flowid=innerFlow -Dazkaban.execid=95 -Dazkaban.jobid=innerJobC -Xms64M -Xmx256M -cp TestAzkabanType.jar com.win.TestAzkabanType innerJobC innerJobA
22-08-2022 16:45:00 CST innerJobC INFO - Environment variables: {JOB_OUTPUT_PROP_FILE=/usr/local/bigdata/azkaban3.51.0/exec-server/azkaban-exec-server-0.1.0-SNAPSHOT/bin/executions/95/innerJobC_output_3166311652967193814_tmp, JOB_PROP_FILE=/usr/local/bigdata/azkaban3.51.0/exec-server/azkaban-exec-server-0.1.0-SNAPSHOT/bin/executions/95/innerJobC_props_1741757618178495101_tmp, KRB5CCNAME=/tmp/krb5cc__mutilexec__innerFlow__innerJobC__95__azkaban, JOB_NAME=innerJobC}
22-08-2022 16:45:00 CST innerJobC INFO - Working directory: /usr/local/bigdata/azkaban3.51.0/exec-server/azkaban-exec-server-0.1.0-SNAPSHOT/bin/executions/95
22-08-2022 16:45:00 CST innerJobC INFO - this is a java class. java type,package:com.win! 参数1=innerJobC 参数2= innerJobA
22-08-2022 16:45:00 CST innerJobC INFO - Process completed successfully in 0 seconds.
22-08-2022 16:45:00 CST innerJobC INFO - output properties file=/usr/local/bigdata/azkaban3.51.0/exec-server/azkaban-exec-server-0.1.0-SNAPSHOT/bin/executions/95/innerJobC_output_3166311652967193814_tmp
22-08-2022 16:45:00 CST innerJobC INFO - Finishing job innerJobC at 1661157900114 with status SUCCEEDED
四、azkaban定时任务调度
配置如下
- 首页选择待处理的project
- 设置好执行计划后
- 执行结果
以上完成了azkaban定时任务配置。
