Deployment took more than 60 seconds, Please check if the requested resources are available in the YARN cluster
在zookeeper正常启动后,运行hadoop,并启动flink发现报错:
2022-10-26 13:24:49,145 INFO org.apache.flink.yarn.YarnClusterDescriptor [] - Deployment took more than 60 seconds. Please check if the requested resources are available in the YARN cluster
网上几乎都说是由于yarn资源不足造成的,经过不断实践,问题解决,解决办法如下:停掉hadoop,修改yarn-site.xml文件为如下:
<configuration>
<!-- Site specific YARN configuration properties -->
<property>
<name>yarn.resourcemanager.hostname</name>
<value>resource主机ip</value>
</property>
<property>
<name>yarn.nodemanager.aux-services</name>
<value>mapreduce_shuffle</value>
</property>
<property>
<name>yarn.nodemanager.env-whitelist</name>
<value>JAVA_HOME,HADOOP_COMMON_HOME,HADOOP_HDFS_HOME,HADOOP_CONF_DIR,CLASSPATH_PREPEND_DISTCACHE,HADOOP_YARN_HOME,HADOOP_MAPRED_HOME</value>
</property>
<property>
<name>yarn.nodemanager.pmem-check-enabled</name>
<value>false</value>
</property>
<property>
<name>yarn.nodemanager.vmem-check-enabled</name>
<value>false</value>
</property>
<property>
<name>yarn.nodemanager.resource.memory-mb</name>
<value>4096</value>
</property>
<!-- 一台NodeManager的总可用(逻辑)cpu核数 -->
<property>
<name>yarn.nodemanager.resource.cpu-vcores</name>
<value>2</value>
</property>
<!-- 容器的虚拟内存使用上限:与物理内存的比率 -->
<property>
<name>yarn.nodemanager.vmem-pmem-ratio</name>
<value>2.1</value>
</property>
</configuration>
保存后,重新启动hadoop及flink发现一切正常。然后运行wordcount.jar一切正常。
附:在UI中运行方法:
上传WordCount.jar,在参数中录入:--input /usr/local/test.text --output /usr/local/output.txt 注意:不需要带file://
而在命令行中,需要带上file://,即:
./flink run /usr/local/WordCount.jar --input file:///usr/local/test.txt --output file:///usr/local/output1.txt

浙公网安备 33010602011771号