hive脚本报错,mark【 Error: Exceeded limits on number of counters - Counters=120 Limit=120】
运行一个hive脚本报错:
[2013-02-25 14:54:00.372]Hadoop job information for Stage-1: number of mappers: 25; number of reducers: 50
[2013-02-25 14:54:00.377][2013-02-25 14:54:00.377] at org.apache.hadoop.mapred.Counters$Group.getCounterForName(Counters.java:312)
[2013-02-25 14:54:00.377] at org.apache.hadoop.mapred.Counters.findCounter(Counters.java:431)
[2013-02-25 14:54:00.377] at org.apache.hadoop.mapred.Counters.getCounter(Counters.java:495)
[2013-02-25 14:54:00.377] at org.apache.hadoop.hive.ql.exec.Operator.updateCounters(Operator.java:1150)
[2013-02-25 14:54:00.377] at org.apache.hadoop.hive.ql.exec.ExecDriver.updateCounters(ExecDriver.java:1000)
[2013-02-25 14:54:00.377] at org.apache.hadoop.hive.ql.exec.HadoopJobExecHelper.updateCounters(HadoopJobExecHelper.java:85)
[2013-02-25 14:54:00.377] at org.apache.hadoop.hive.ql.exec.HadoopJobExecHelper.progress(HadoopJobExecHelper.java:312)
[2013-02-25 14:54:00.378] at org.apache.hadoop.hive.ql.exec.HadoopJobExecHelper.progress(HadoopJobExecHelper.java:685)
[2013-02-25 14:54:00.378] at org.apache.hadoop.hive.ql.exec.ExecDriver.execute(ExecDriver.java:458)
[2013-02-25 14:54:00.378] at org.apache.hadoop.hive.ql.exec.MapRedTask.execute(MapRedTask.java:136)
[2013-02-25 14:54:00.378] at org.apache.hadoop.hive.ql.exec.Task.executeTask(Task.java:133)
[2013-02-25 14:54:00.378] at org.apache.hadoop.hive.ql.exec.TaskRunner.runSequential(TaskRunner.java:57)
[2013-02-25 14:54:00.378] at org.apache.hadoop.hive.ql.exec.TaskRunner.run(TaskRunner.java:47)
[2013-02-25 14:54:00.378]Ended Job = job_201301181421_100273 with exception 'org.apache.hadoop.mapred.Counters$CountersExceededException(Error: Exceeded limits on
number of counters - Counters=120 Limit=120)'
[2013-02-25 14:54:02.122]FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.MapRedTask
解决:
./conf/mapred-site.xml 追加:
<property>
<name>mapreduce.job.counters.limit</name>
<value>1400</value>
</property>
posted on 2016-05-21 21:11 1130136248 阅读(363) 评论(0) 收藏 举报
浙公网安备 33010602011771号