使用YARN命令管理Hadoop作业
使用YARN命令管理Hadoop作业
作者：尹正杰
版权声明：原创作品，谢绝转载！否则将追究法律责任。
一.YARN命令概述
如下图所示，Hadoop的yarn命令可以管理YARN的各个方面。虽然ResourceManager Web UI非常有用，但当不能使用Web UI时，就需要使用命令来处理问题。 Hadoop的yarn命令具有广泛的使用范围，从某种意义上来说，它可以帮助我们管理大量的Hadoop任务，例如获取和杀死正在运行的应用程序，获取作业和守护程序日志，甚至管理ResourceManager的上下线。 可以使用yarn命令从命令行监视和管理应用程序。例如，可以使用"yarn application -list running"命令查看正在运行的作业的状态。 同样，可以使用"yarn application -kill <Application ID>"命令从命令行正常删除正在运行的应用程序，而不是使用Linux kill命令杀死进程。
最好用的yarn命令是application子命令，它允许快速查看某个时间点的集群时间点的使用情况。 如下图所示，yarn top命令提供了有关"正在运行(running)","已提交(submitted)","待定(pending)","已完成(completed)","已终止(killed)","失败(failed)"的摘要信息，还显示为集群的足有作业分配的内存量和CPU内核量。
本篇博客的重点是监控和管理集群，我们需要了解如何使用yarn命令监控YARN集群。 通过该命令可以查看基本作业信息并执行其他任务，例如"yarn application -kill application_1604218163813_0003"来杀死Application ID为"application_1604218163813_0003"正在运行的Job。 使用"yarn application"命令可以执行以下管理任务: (1)列出集群中运行的应用程序; (2)杀死正在运行的应用程序; (3)获取正在运行的应用程序的状态;
二.查看YARN应用程序
1>."yarn application -list"命令
如下图所示，可以使用"yarn application -list"命令，立即查看集群中的所有应用程序(我们称为作业，YARN称为应用)。
[root@hadoop101.yinzhengjie.com ~]# yarn application -list 20/11/01 17:06:30 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable 20/11/01 17:06:30 INFO client.RMProxy: Connecting to ResourceManager at hadoop101.yinzhengjie.com/172.200.6.101:8032 Total number of applications (application-types: [], states: [SUBMITTED, ACCEPTED, RUNNING] and tags: []):2 Application-Id Application-Name Application-Type User Queue State Final-State Progress Tracking-URLapplication_1604218163813_0007 word count MAPREDUCE root root.yinzhengjie.operation.op_queue01 RUNNING UNDEFINED 50% http://hadoop104.yinzhengjie.com:8705application_1604218163813_0008 word count MAPREDUCE root root.yinzhengjie.development ACCEPTED UNDEFINED 0% N/A[root@hadoop101.yinzhengjie.com ~]# [root@hadoop101.yinzhengjie.com ~]#
2>."yarn application -list -appStates <States>"命令
使用"yarn application -list"命令可以检索所有作业的列表，而不考虑它们的状态。 作业可以具有以下状态： "所有(ALL)","新的(NEW)","新的保存(NEW_SAVING)","提交(SUBMITTED)","接受(ACCEPTED)","正在运行(RUNNING)","已完成(FINISHED)","FAILED(失败)"和"被杀死的(KILLED)"。
[root@hadoop101.yinzhengjie.com ~]# yarn application -list -appStates all 20/11/01 17:17:23 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable 20/11/01 17:17:24 INFO client.RMProxy: Connecting to ResourceManager at hadoop101.yinzhengjie.com/172.200.6.101:8032 Total number of applications (application-types: [], states: [NEW, NEW_SAVING, SUBMITTED, ACCEPTED, RUNNING, FINISHED, FAILED, KILLED] and tags: []):8 Application-Id Application-Name Application-Type User Queue State Final-State Progress Tracking-URL application_1604218163813_0007 word count MAPREDUCE root root.yinzhengjie.operation.op_queue01 FINISHED SUCCEEDED 100% http://hadoop105.yinzhengjie.com:19888/jobhistory/job/job_1604218163813_0007 application_1604218163813_0008 word count MAPREDUCE root root.yinzhengjie.development FINISHED SUCCEEDED 100% http://hadoop105.yinzhengjie.com:19888/jobhistory/job/job_1604218163813_0008 application_1604218163813_0003 word count MAPREDUCE root root.yinzhengjie.operation.op_queue01 KILLED KILLED 100% http://hadoop101.yinzhengjie.com:8088/cluster/app/application_1604218163813_0003 application_1604218163813_0004 word count MAPREDUCE root root.yinzhengjie.operation.op_queue01 FINISHED SUCCEEDED 100% http://hadoop105.yinzhengjie.com:19888/jobhistory/job/job_1604218163813_0004 application_1604218163813_0005 word count MAPREDUCE root root.yinzhengjie.operation.op_queue01 FINISHED SUCCEEDED 100% http://hadoop105.yinzhengjie.com:19888/jobhistory/job/job_1604218163813_0005 application_1604218163813_0006 word count MAPREDUCE root root.yinzhengjie.operation.op_queue01 FINISHED SUCCEEDED 100% http://hadoop105.yinzhengjie.com:19888/jobhistory/job/job_1604218163813_0006 application_1604218163813_0001 word count MAPREDUCE root root.yinzhengjie.operation.op_queue01 FINISHED SUCCEEDED 100% http://hadoop105.yinzhengjie.com:19888/jobhistory/job/job_1604218163813_0001 application_1604218163813_0002 word count MAPREDUCE root root.yinzhengjie.operation.op_queue01 FINISHED SUCCEEDED 100% http://hadoop105.yinzhengjie.com:19888/jobhistory/job/job_1604218163813_0002 [root@hadoop101.yinzhengjie.com ~]#
[root@hadoop101.yinzhengjie.com ~]# yarn application -list -appStates all 20/11/01 17:17:23 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable 20/11/01 17:17:24 INFO client.RMProxy: Connecting to ResourceManager at hadoop101.yinzhengjie.com/172.200.6.101:8032 Total number of applications (application-types: [], states: [NEW, NEW_SAVING, SUBMITTED, ACCEPTED, RUNNING, FINISHED, FAILED, KILLED] and tags: []):8 Application-Id Application-Name Application-Type User Queue State Final-State Progress Tracking-URL application_1604218163813_0007 word count MAPREDUCE root root.yinzhengjie.operation.op_queue01 FINISHED SUCCEEDED 100% http://hadoop105.yinzhengjie.com:19888/jobhistory/job/job_1604218163813_0007 application_1604218163813_0008 word count MAPREDUCE root root.yinzhengjie.development FINISHED SUCCEEDED 100% http://hadoop105.yinzhengjie.com:19888/jobhistory/job/job_1604218163813_0008 application_1604218163813_0003 word count MAPREDUCE root root.yinzhengjie.operation.op_queue01 KILLED KILLED 100% http://hadoop101.yinzhengjie.com:8088/cluster/app/application_1604218163813_0003 application_1604218163813_0004 word count MAPREDUCE root root.yinzhengjie.operation.op_queue01 FINISHED SUCCEEDED 100% http://hadoop105.yinzhengjie.com:19888/jobhistory/job/job_1604218163813_0004 application_1604218163813_0005 word count MAPREDUCE root root.yinzhengjie.operation.op_queue01 FINISHED SUCCEEDED 100% http://hadoop105.yinzhengjie.com:19888/jobhistory/job/job_1604218163813_0005 application_1604218163813_0006 word count MAPREDUCE root root.yinzhengjie.operation.op_queue01 FINISHED SUCCEEDED 100% http://hadoop105.yinzhengjie.com:19888/jobhistory/job/job_1604218163813_0006 application_1604218163813_0001 word count MAPREDUCE root root.yinzhengjie.operation.op_queue01 FINISHED SUCCEEDED 100% http://hadoop105.yinzhengjie.com:19888/jobhistory/job/job_1604218163813_0001 application_1604218163813_0002 word count MAPREDUCE root root.yinzhengjie.operation.op_queue01 FINISHED SUCCEEDED 100% http://hadoop105.yinzhengjie.com:19888/jobhistory/job/job_1604218163813_0002 [root@hadoop101.yinzhengjie.com ~]# [root@hadoop101.yinzhengjie.com ~]# yarn application -list -appStates killed 20/11/01 17:17:27 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable 20/11/01 17:17:27 INFO client.RMProxy: Connecting to ResourceManager at hadoop101.yinzhengjie.com/172.200.6.101:8032 Total number of applications (application-types: [], states: [KILLED] and tags: []):1 Application-Id Application-Name Application-Type User Queue State Final-State Progress Tracking-URL application_1604218163813_0003 word count MAPREDUCE root root.yinzhengjie.operation.op_queue01 KILLED KILLED 100% http://hadoop101.yinzhengjie.com:8088/cluster/app/application_1604218163813_0003 [root@hadoop101.yinzhengjie.com ~]#
[root@hadoop101.yinzhengjie.com ~]# yarn application -list -appStates all 20/11/01 17:17:23 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable 20/11/01 17:17:24 INFO client.RMProxy: Connecting to ResourceManager at hadoop101.yinzhengjie.com/172.200.6.101:8032 Total number of applications (application-types: [], states: [NEW, NEW_SAVING, SUBMITTED, ACCEPTED, RUNNING, FINISHED, FAILED, KILLED] and tags: []):8 Application-Id Application-Name Application-Type User Queue State Final-State Progress Tracking-URL application_1604218163813_0007 word count MAPREDUCE root root.yinzhengjie.operation.op_queue01 FINISHED SUCCEEDED 100% http://hadoop105.yinzhengjie.com:19888/jobhistory/job/job_1604218163813_0007 application_1604218163813_0008 word count MAPREDUCE root root.yinzhengjie.development FINISHED SUCCEEDED 100% http://hadoop105.yinzhengjie.com:19888/jobhistory/job/job_1604218163813_0008 application_1604218163813_0003 word count MAPREDUCE root root.yinzhengjie.operation.op_queue01 KILLED KILLED 100% http://hadoop101.yinzhengjie.com:8088/cluster/app/application_1604218163813_0003 application_1604218163813_0004 word count MAPREDUCE root root.yinzhengjie.operation.op_queue01 FINISHED SUCCEEDED 100% http://hadoop105.yinzhengjie.com:19888/jobhistory/job/job_1604218163813_0004 application_1604218163813_0005 word count MAPREDUCE root root.yinzhengjie.operation.op_queue01 FINISHED SUCCEEDED 100% http://hadoop105.yinzhengjie.com:19888/jobhistory/job/job_1604218163813_0005 application_1604218163813_0006 word count MAPREDUCE root root.yinzhengjie.operation.op_queue01 FINISHED SUCCEEDED 100% http://hadoop105.yinzhengjie.com:19888/jobhistory/job/job_1604218163813_0006 application_1604218163813_0001 word count MAPREDUCE root root.yinzhengjie.operation.op_queue01 FINISHED SUCCEEDED 100% http://hadoop105.yinzhengjie.com:19888/jobhistory/job/job_1604218163813_0001 application_1604218163813_0002 word count MAPREDUCE root root.yinzhengjie.operation.op_queue01 FINISHED SUCCEEDED 100% http://hadoop105.yinzhengjie.com:19888/jobhistory/job/job_1604218163813_0002 [root@hadoop101.yinzhengjie.com ~]# [root@hadoop101.yinzhengjie.com ~]# yarn application -list -appStates failed 20/11/01 17:17:32 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable 20/11/01 17:17:32 INFO client.RMProxy: Connecting to ResourceManager at hadoop101.yinzhengjie.com/172.200.6.101:8032 Total number of applications (application-types: [], states: [FAILED] and tags: []):0 Application-Id Application-Name Application-Type User Queue State Final-State Progress Tracking-URL [root@hadoop101.yinzhengjie.com ~]#
[root@hadoop101.yinzhengjie.com ~]# yarn application -list -appStates all 20/11/01 17:17:23 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable 20/11/01 17:17:24 INFO client.RMProxy: Connecting to ResourceManager at hadoop101.yinzhengjie.com/172.200.6.101:8032 Total number of applications (application-types: [], states: [NEW, NEW_SAVING, SUBMITTED, ACCEPTED, RUNNING, FINISHED, FAILED, KILLED] and tags: []):8 Application-Id Application-Name Application-Type User Queue State Final-State Progress Tracking-URL application_1604218163813_0007 word count MAPREDUCE root root.yinzhengjie.operation.op_queue01 FINISHED SUCCEEDED 100% http://hadoop105.yinzhengjie.com:19888/jobhistory/job/job_1604218163813_0007 application_1604218163813_0008 word count MAPREDUCE root root.yinzhengjie.development FINISHED SUCCEEDED 100% http://hadoop105.yinzhengjie.com:19888/jobhistory/job/job_1604218163813_0008 application_1604218163813_0003 word count MAPREDUCE root root.yinzhengjie.operation.op_queue01 KILLED KILLED 100% http://hadoop101.yinzhengjie.com:8088/cluster/app/application_1604218163813_0003 application_1604218163813_0004 word count MAPREDUCE root root.yinzhengjie.operation.op_queue01 FINISHED SUCCEEDED 100% http://hadoop105.yinzhengjie.com:19888/jobhistory/job/job_1604218163813_0004 application_1604218163813_0005 word count MAPREDUCE root root.yinzhengjie.operation.op_queue01 FINISHED SUCCEEDED 100% http://hadoop105.yinzhengjie.com:19888/jobhistory/job/job_1604218163813_0005 application_1604218163813_0006 word count MAPREDUCE root root.yinzhengjie.operation.op_queue01 FINISHED SUCCEEDED 100% http://hadoop105.yinzhengjie.com:19888/jobhistory/job/job_1604218163813_0006 application_1604218163813_0001 word count MAPREDUCE root root.yinzhengjie.operation.op_queue01 FINISHED SUCCEEDED 100% http://hadoop105.yinzhengjie.com:19888/jobhistory/job/job_1604218163813_0001 application_1604218163813_0002 word count MAPREDUCE root root.yinzhengjie.operation.op_queue01 FINISHED SUCCEEDED 100% http://hadoop105.yinzhengjie.com:19888/jobhistory/job/job_1604218163813_0002 [root@hadoop101.yinzhengjie.com ~]# [root@hadoop101.yinzhengjie.com ~]# yarn application -list -appStates finished 20/11/01 17:25:04 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable 20/11/01 17:25:04 INFO client.RMProxy: Connecting to ResourceManager at hadoop101.yinzhengjie.com/172.200.6.101:8032 Total number of applications (application-types: [], states: [FINISHED] and tags: []):7 Application-Id Application-Name Application-Type User Queue State Final-State Progress Tracking-URL application_1604218163813_0007 word count MAPREDUCE root root.yinzhengjie.operation.op_queue01 FINISHED SUCCEEDED 100% http://hadoop105.yinzhengjie.com:19888/jobhistory/job/job_1604218163813_0007 application_1604218163813_0008 word count MAPREDUCE root root.yinzhengjie.development FINISHED SUCCEEDED 100% http://hadoop105.yinzhengjie.com:19888/jobhistory/job/job_1604218163813_0008 application_1604218163813_0004 word count MAPREDUCE root root.yinzhengjie.operation.op_queue01 FINISHED SUCCEEDED 100% http://hadoop105.yinzhengjie.com:19888/jobhistory/job/job_1604218163813_0004 application_1604218163813_0005 word count MAPREDUCE root root.yinzhengjie.operation.op_queue01 FINISHED SUCCEEDED 100% http://hadoop105.yinzhengjie.com:19888/jobhistory/job/job_1604218163813_0005 application_1604218163813_0006 word count MAPREDUCE root root.yinzhengjie.operation.op_queue01 FINISHED SUCCEEDED 100% http://hadoop105.yinzhengjie.com:19888/jobhistory/job/job_1604218163813_0006 application_1604218163813_0001 word count MAPREDUCE root root.yinzhengjie.operation.op_queue01 FINISHED SUCCEEDED 100% http://hadoop105.yinzhengjie.com:19888/jobhistory/job/job_1604218163813_0001 application_1604218163813_0002 word count MAPREDUCE root root.yinzhengjie.operation.op_queue01 FINISHED SUCCEEDED 100% http://hadoop105.yinzhengjie.com:19888/jobhistory/job/job_1604218163813_0002 [root@hadoop101.yinzhengjie.com ~]#
三.检查应用程序的状态
1>.通过"yarn application -status <Application ID>"查看特定作业的状态
[root@hadoop101.yinzhengjie.com ~]# yarn application -list -appStates all 20/11/01 17:17:23 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable 20/11/01 17:17:24 INFO client.RMProxy: Connecting to ResourceManager at hadoop101.yinzhengjie.com/172.200.6.101:8032 Total number of applications (application-types: [], states: [NEW, NEW_SAVING, SUBMITTED, ACCEPTED, RUNNING, FINISHED, FAILED, KILLED] and tags: []):8 Application-Id Application-Name Application-Type User Queue State Final-State Progress Tracking-URL application_1604218163813_0007 word count MAPREDUCE root root.yinzhengjie.operation.op_queue01 FINISHED SUCCEEDED 100% http://hadoop105.yinzhengjie.com:19888/jobhistory/job/job_1604218163813_0007 application_1604218163813_0008 word count MAPREDUCE root root.yinzhengjie.development FINISHED SUCCEEDED 100% http://hadoop105.yinzhengjie.com:19888/jobhistory/job/job_1604218163813_0008 application_1604218163813_0003 word count MAPREDUCE root root.yinzhengjie.operation.op_queue01 KILLED KILLED 100% http://hadoop101.yinzhengjie.com:8088/cluster/app/application_1604218163813_0003 application_1604218163813_0004 word count MAPREDUCE root root.yinzhengjie.operation.op_queue01 FINISHED SUCCEEDED 100% http://hadoop105.yinzhengjie.com:19888/jobhistory/job/job_1604218163813_0004 application_1604218163813_0005 word count MAPREDUCE root root.yinzhengjie.operation.op_queue01 FINISHED SUCCEEDED 100% http://hadoop105.yinzhengjie.com:19888/jobhistory/job/job_1604218163813_0005 application_1604218163813_0006 word count MAPREDUCE root root.yinzhengjie.operation.op_queue01 FINISHED SUCCEEDED 100% http://hadoop105.yinzhengjie.com:19888/jobhistory/job/job_1604218163813_0006 application_1604218163813_0001 word count MAPREDUCE root root.yinzhengjie.operation.op_queue01 FINISHED SUCCEEDED 100% http://hadoop105.yinzhengjie.com:19888/jobhistory/job/job_1604218163813_0001 application_1604218163813_0002 word count MAPREDUCE root root.yinzhengjie.operation.op_queue01 FINISHED SUCCEEDED 100% http://hadoop105.yinzhengjie.com:19888/jobhistory/job/job_1604218163813_0002 [root@hadoop101.yinzhengjie.com ~]# [root@hadoop101.yinzhengjie.com ~]# yarn application -status application_1604218163813_0008 20/11/01 17:33:54 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable 20/11/01 17:33:54 INFO client.RMProxy: Connecting to ResourceManager at hadoop101.yinzhengjie.com/172.200.6.101:8032 20/11/01 17:33:55 INFO conf.Configuration: resource-types.xml not found 20/11/01 17:33:55 INFO resource.ResourceUtils: Unable to find 'resource-types.xml'. 20/11/01 17:33:55 INFO resource.ResourceUtils: Adding resource type - name = memory-mb, units = Mi, type = COUNTABLE 20/11/01 17:33:55 INFO resource.ResourceUtils: Adding resource type - name = vcores, units = , type = COUNTABLE Application Report : Application-Id : application_1604218163813_0008 Application-Name : word count Application-Type : MAPREDUCE User : root Queue : root.yinzhengjie.development Application Priority : 0 Start-Time : 1604221581251 Finish-Time : 1604221620578 Progress : 100% State : FINISHED Final-State : SUCCEEDED Tracking-URL : http://hadoop105.yinzhengjie.com:19888/jobhistory/job/job_1604218163813_0008 RPC Port : 2890 AM Host : hadoop103.yinzhengjie.com Aggregate Resource Allocation : 58152 MB-seconds, 27 vcore-seconds Aggregate Resource Preempted : 0 MB-seconds, 0 vcore-seconds Log Aggregation Status : SUCCEEDED Diagnostics : Unmanaged Application : false Application Node Label Expression : <Not set> AM container Node Label Expression : <DEFAULT_PARTITION> TimeoutType : LIFETIME ExpiryTime : UNLIMITED RemainingTime : -1seconds [root@hadoop101.yinzhengjie.com ~]#
2>."yarn applicationattempt -list <Application ID>"命令
[root@hadoop101.yinzhengjie.com ~]# yarn application -list -appStates all 20/11/01 17:17:23 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable 20/11/01 17:17:24 INFO client.RMProxy: Connecting to ResourceManager at hadoop101.yinzhengjie.com/172.200.6.101:8032 Total number of applications (application-types: [], states: [NEW, NEW_SAVING, SUBMITTED, ACCEPTED, RUNNING, FINISHED, FAILED, KILLED] and tags: []):8 Application-Id Application-Name Application-Type User Queue State Final-State Progress Tracking-URL application_1604218163813_0007 word count MAPREDUCE root root.yinzhengjie.operation.op_queue01 FINISHED SUCCEEDED 100% http://hadoop105.yinzhengjie.com:19888/jobhistory/job/job_1604218163813_0007 application_1604218163813_0008 word count MAPREDUCE root root.yinzhengjie.development FINISHED SUCCEEDED 100% http://hadoop105.yinzhengjie.com:19888/jobhistory/job/job_1604218163813_0008 application_1604218163813_0003 word count MAPREDUCE root root.yinzhengjie.operation.op_queue01 KILLED KILLED 100% http://hadoop101.yinzhengjie.com:8088/cluster/app/application_1604218163813_0003 application_1604218163813_0004 word count MAPREDUCE root root.yinzhengjie.operation.op_queue01 FINISHED SUCCEEDED 100% http://hadoop105.yinzhengjie.com:19888/jobhistory/job/job_1604218163813_0004 application_1604218163813_0005 word count MAPREDUCE root root.yinzhengjie.operation.op_queue01 FINISHED SUCCEEDED 100% http://hadoop105.yinzhengjie.com:19888/jobhistory/job/job_1604218163813_0005 application_1604218163813_0006 word count MAPREDUCE root root.yinzhengjie.operation.op_queue01 FINISHED SUCCEEDED 100% http://hadoop105.yinzhengjie.com:19888/jobhistory/job/job_1604218163813_0006 application_1604218163813_0001 word count MAPREDUCE root root.yinzhengjie.operation.op_queue01 FINISHED SUCCEEDED 100% http://hadoop105.yinzhengjie.com:19888/jobhistory/job/job_1604218163813_0001 application_1604218163813_0002 word count MAPREDUCE root root.yinzhengjie.operation.op_queue01 FINISHED SUCCEEDED 100% http://hadoop105.yinzhengjie.com:19888/jobhistory/job/job_1604218163813_0002 [root@hadoop101.yinzhengjie.com ~]# [root@hadoop101.yinzhengjie.com ~]# yarn applicationattempt -list application_1604218163813_0008 20/11/01 17:44:51 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable 20/11/01 17:44:51 INFO client.RMProxy: Connecting to ResourceManager at hadoop101.yinzhengjie.com/172.200.6.101:8032 Total number of application attempts :1 ApplicationAttempt-Id State AM-Container-Id Tracking-URL appattempt_1604218163813_0008_000001 FINISHED container_1604218163813_0008_01_000001 http://hadoop101.yinzhengjie.com:8088/proxy/application_1604218163813_0008/ [root@hadoop101.yinzhengjie.com ~]#
3>."yarn applicationattempt -status <Application Attempt ID>"命令
[root@hadoop101.yinzhengjie.com ~]# yarn applicationattempt -list application_1604218163813_0008 20/11/01 17:51:51 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable 20/11/01 17:51:51 INFO client.RMProxy: Connecting to ResourceManager at hadoop101.yinzhengjie.com/172.200.6.101:8032 Total number of application attempts :1 ApplicationAttempt-Id State AM-Container-Id Tracking-URL appattempt_1604218163813_0008_000001 FINISHED container_1604218163813_0008_01_000001 http://hadoop101.yinzhengjie.com:8088/proxy/application_1604218163813_0008/ [root@hadoop101.yinzhengjie.com ~]# [root@hadoop101.yinzhengjie.com ~]# [root@hadoop101.yinzhengjie.com ~]# yarn applicationattempt -status appattempt_1604218163813_0008_000001 20/11/01 17:52:02 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable 20/11/01 17:52:02 INFO client.RMProxy: Connecting to ResourceManager at hadoop101.yinzhengjie.com/172.200.6.101:8032 Application Attempt Report : ApplicationAttempt-Id : appattempt_1604218163813_0008_000001 State : FINISHED AMContainer : container_1604218163813_0008_01_000001 Tracking-URL : http://hadoop101.yinzhengjie.com:8088/proxy/application_1604218163813_0008/ RPC Port : 2890 AM Host : hadoop103.yinzhengjie.com Diagnostics : [root@hadoop101.yinzhengjie.com ~]# [root@hadoop101.yinzhengjie.com ~]#
四.kill正在执行的作业("yarn application -kill <Application ID>")
[root@hadoop101.yinzhengjie.com ~]# yarn application -list 20/11/01 17:59:08 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable 20/11/01 17:59:08 INFO client.RMProxy: Connecting to ResourceManager at hadoop101.yinzhengjie.com/172.200.6.101:8032 Total number of applications (application-types: [], states: [SUBMITTED, ACCEPTED, RUNNING] and tags: []):1 Application-Id Application-Name Application-Type User Queue State Final-State Progress Tracking-URL application_1604218163813_0013 word count MAPREDUCE root root.yinzhengjie.testing RUNNING UNDEFINED 5% http://hadoop104.yinzhengjie.com:2692 [root@hadoop101.yinzhengjie.com ~]# [root@hadoop101.yinzhengjie.com ~]# yarn application -kill application_1604218163813_0013 20/11/01 17:59:17 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable 20/11/01 17:59:18 INFO client.RMProxy: Connecting to ResourceManager at hadoop101.yinzhengjie.com/172.200.6.101:8032 Killing application application_1604218163813_0013 20/11/01 17:59:18 INFO impl.YarnClientImpl: Killed application application_1604218163813_0013 [root@hadoop101.yinzhengjie.com ~]# [root@hadoop101.yinzhengjie.com ~]# yarn application -list 20/11/01 17:59:22 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable 20/11/01 17:59:22 INFO client.RMProxy: Connecting to ResourceManager at hadoop101.yinzhengjie.com/172.200.6.101:8032 Total number of applications (application-types: [], states: [SUBMITTED, ACCEPTED, RUNNING] and tags: []):0 Application-Id Application-Name Application-Type User Queue State Final-State Progress Tracking-URL [root@hadoop101.yinzhengjie.com ~]# [root@hadoop101.yinzhengjie.com ~]# [root@hadoop101.yinzhengjie.com ~]# yarn application -list -appStates killed 20/11/01 17:59:27 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable 20/11/01 17:59:27 INFO client.RMProxy: Connecting to ResourceManager at hadoop101.yinzhengjie.com/172.200.6.101:8032 Total number of applications (application-types: [], states: [KILLED] and tags: []):2 Application-Id Application-Name Application-Type User Queue State Final-State Progress Tracking-URL application_1604218163813_0013 word count MAPREDUCE root root.yinzhengjie.testing KILLED KILLED 10 0% http://hadoop101.yinzhengjie.com:8088/cluster/app/application_1604218163813_0013 application_1604218163813_0003 word count MAPREDUCE root root.yinzhengjie.operation.op_queue01 KILLED KILLED 100% http://hadoop101.yinzhengjie.com:8088/cluster/app/application_1604218163813_0003 [root@hadoop101.yinzhengjie.com ~]#
五.检查节点状态
[root@hadoop101.yinzhengjie.com ~]# yarn node -all -list 20/11/01 18:09:42 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable 20/11/01 18:09:42 INFO client.RMProxy: Connecting to ResourceManager at hadoop101.yinzhengjie.com/172.200.6.101:8032 Total Nodes:3 Node-Id Node-State Node-Http-Address Number-of-Running-Containers hadoop104.yinzhengjie.com:6666 RUNNING hadoop104.yinzhengjie.com:8042 0 hadoop102.yinzhengjie.com:6666 RUNNING hadoop102.yinzhengjie.com:8042 0 hadoop103.yinzhengjie.com:6666 RUNNING hadoop103.yinzhengjie.com:8042 0 [root@hadoop101.yinzhengjie.com ~]# [root@hadoop101.yinzhengjie.com ~]#
六.检查YARN的队列状态
[root@hadoop101.yinzhengjie.com ~]# yarn queue -status yinzhengjie 20/11/01 18:15:47 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable 20/11/01 18:15:48 INFO client.RMProxy: Connecting to ResourceManager at hadoop101.yinzhengjie.com/172.200.6.101:8032 Queue Information : Queue Name : root.yinzhengjie State : RUNNING Capacity : .0% Current Capacity : .0% Maximum Capacity : -100.0% Default Node Label expression : <DEFAULT_PARTITION> Accessible Node Labels : [root@hadoop101.yinzhengjie.com ~]#
七.获取作业的日志
[root@hadoop101.yinzhengjie.com ~]# ll total 0 [root@hadoop101.yinzhengjie.com ~]# [root@hadoop101.yinzhengjie.com ~]# yarn logs -applicationId application_1604218163813_0008 > application_1604218163813_0008.log 20/11/01 18:27:09 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable 20/11/01 18:27:09 INFO client.RMProxy: Connecting to ResourceManager at hadoop101.yinzhengjie.com/172.200.6.101:8032 [root@hadoop101.yinzhengjie.com ~]# [root@hadoop101.yinzhengjie.com ~]# ll total 140 -rw-r--r-- 1 root root 140715 Nov 1 18:27 application_1604218163813_0008.log [root@hadoop101.yinzhengjie.com ~]# [root@hadoop101.yinzhengjie.com ~]# [root@hadoop101.yinzhengjie.com ~]# ll -h total 140K -rw-r--r-- 1 root root 138K Nov 1 18:27 application_1604218163813_0008.log [root@hadoop101.yinzhengjie.com ~]#
八.YARN管理命令
1>.节点的上线与下线
2>.高可用ResourceManager
3>.其他
当你的才华还撑不起你的野心的时候，你就应该静下心来学习。当你的能力还驾驭不了你的目标的时候，你就应该沉下心来历练。问问自己，想要怎样的人生。
欢迎加入基础架构自动化运维：598432640，大数据SRE进阶之路：959042252，DevOps进阶之路：526991186
