Hive 程序内存溢出错误分析
原hive程序：
|
INSERT OVERWRITE TABLE MID_RES_LINK_LGC_BT PARTITION ( local_area_id = 20, dt = '20130831' ) SELECT LL.LGC_LINE_ID, LL.RES_SPEC_ID, T.LINK_ID, LP2LA.PHY_EQP_ID, PEA.RES_SPEC_ID, LL2PA.PHY_PORT_ID, PTPA.RES_SPEC_ID, LP2LZ.PHY_EQP_ID, PEZ.RES_SPEC_ID, LL2PZ.PHY_PORT_ID, PTPZ.RES_SPEC_ID, T.A_LGC_EQP_ID, LEA.RES_SPEC_ID, T.A_LGC_PORT_ID, LPA.RES_SPEC_ID, T.Z_LGC_EQP_ID, LEZ.RES_SPEC_ID, T.Z_LGC_PORT_ID, LPZ.RES_SPEC_ID FROM TC_RES_LNK_LGC_LINK_COMPONENT T JOIN TC_RES_LGC_LINE LL ON ( T.LINE_ID = LL.LGC_LINE_ID AND LL.LOCAL_AREA_ID = 20 AND LL.DT = '20130801' AND LL.RES_SPEC_ID IN ( 1744, 1745, 1749, 2929, 2931, 2941, 2943 ) AND T.LOCAL_AREA_ID = 20 AND T.DT = '20130801' ) JOIN TC_RES_LGC_PHYEQP_2_LGCEQP LP2LA ON ( LP2LA.LGC_EQP_ID = T.A_LGC_EQP_ID AND LP2LA.LOCAL_AREA_ID = 20 AND LP2LA.DT = '20130801' AND T.LOCAL_AREA_ID = 20 AND T.DT = '20130801' ) JOIN TC_RES_PHY_EQUIPMENT PEA ON ( LP2LA.PHY_EQP_ID = PEA.PHY_EQP_ID AND LP2LA.LOCAL_AREA_ID = 20 AND LP2LA.DT = '20130801' AND PEA.LOCAL_AREA_ID = 20 AND PEA.DT = '20130801' ) JOIN TC_RES_LGC_LGCPORT_2_PHYPORT LL2PA ON ( LL2PA.LGC_PORT_ID = T.A_LGC_PORT_ID AND LL2PA.LOCAL_AREA_ID = 20 AND LL2PA.DT = '20130801' AND T.LOCAL_AREA_ID = 20 AND T.DT = '20130801' ) JOIN TC_RES_PHY_TERM_PORT PTPA ON ( PTPA.PHY_PORT_ID = LL2PA.PHY_PORT_ID AND PTPA.LOCAL_AREA_ID = 20 AND PTPA.DT = '20130801' AND LL2PA.LOCAL_AREA_ID = 20 AND LL2PA.DT = '20130801' ) JOIN TC_RES_LGC_PHYEQP_2_LGCEQP LP2LZ ON ( LP2LZ.LGC_EQP_ID = T.Z_LGC_EQP_ID AND LP2LZ.LOCAL_AREA_ID = 20 AND LP2LZ.DT = '20130801' AND T.LOCAL_AREA_ID = 20 AND T.DT = '20130801' ) JOIN TC_RES_PHY_EQUIPMENT PEZ ON ( LP2LZ.PHY_EQP_ID = PEZ.PHY_EQP_ID AND LP2LZ.LOCAL_AREA_ID = 20 AND LP2LZ.DT = '20130801' AND PEZ.LOCAL_AREA_ID = 20 AND PEZ.DT = '20130801' ) JOIN TC_RES_LGC_LGCPORT_2_PHYPORT LL2PZ ON ( LL2PZ.LGC_PORT_ID = T.Z_LGC_PORT_ID AND LL2PZ.LOCAL_AREA_ID = 20 AND LL2PZ.DT = '20130801' AND T.LOCAL_AREA_ID = 20 AND T.DT = '20130801' ) JOIN TC_RES_PHY_TERM_PORT PTPZ ON ( PTPZ.PHY_PORT_ID = LL2PZ.PHY_PORT_ID AND PTPZ.LOCAL_AREA_ID = 20 AND PTPZ.DT = '20130801' AND LL2PZ.LOCAL_AREA_ID = 20 AND LL2PZ.DT = '20130801' ) JOIN TC_RES_LGC_EQUIPMENT LEA ON ( LEA.LGC_EQP_ID = T.A_LGC_EQP_ID AND LEA.LOCAL_AREA_ID = 20 AND LEA.DT = '20130801' AND T.LOCAL_AREA_ID = 20 AND T.DT = '20130801' ) JOIN TC_RES_LGC_PORT LPA ON ( LPA.LGC_PORT_ID = T.A_LGC_PORT_ID AND LPA.LOCAL_AREA_ID = 20 AND LPA.DT = '20130801' AND T.LOCAL_AREA_ID = 20 AND T.DT = '20130801' ) JOIN TC_RES_LGC_EQUIPMENT LEZ ON ( LEZ.LGC_EQP_ID = T.Z_LGC_EQP_ID AND LEA.LOCAL_AREA_ID = 20 AND LEA.DT = '20130801' AND T.LOCAL_AREA_ID = 20 AND T.DT = '20130801' ) JOIN TC_RES_LGC_PORT LPZ ON ( LPZ.LGC_PORT_ID = T.Z_LGC_PORT_ID AND LPA.LOCAL_AREA_ID = 20 AND LPA.DT = '20130801' AND T.LOCAL_AREA_ID = 20 AND T.DT = '20130801' )
|
生成执行计划
|
STAGE DEPENDENCIES: Stage-1 is a root stage Stage-2 depends on stages: Stage-1 Stage-3 depends on stages: Stage-2 Stage-4 depends on stages: Stage-3 Stage-5 depends on stages: Stage-4 Stage-6 depends on stages: Stage-5 Stage-7 depends on stages: Stage-6 Stage-8 depends on stages: Stage-7 Stage-9 depends on stages: Stage-8 Stage-0 depends on stages: Stage-9 Stage-10 depends on stages: Stage-0
STAGE PLANS: Stage: Stage-1 Map Reduce Alias -> Map Operator Tree: ll TableScan alias: ll Filter Operator predicate: expr: (((local_area_id = 20.0) and (dt = '20130801')) and (res_spec_id) IN (1744, 1745, 1749, 2929, 2931, 2941, 2943)) type: boolean Reduce Output Operator key expressions: expr: lgc_line_id type: bigint sort order: + Map-reduce partition columns: expr: lgc_line_id type: bigint tag: 1 value expressions: expr: lgc_line_id type: bigint expr: res_spec_id type: int t TableScan alias: t Filter Operator predicate: expr: ((local_area_id = 20.0) and (dt = '20130801')) type: boolean Reduce Output Operator key expressions: expr: line_id type: bigint sort order: + Map-reduce partition columns: expr: line_id type: bigint tag: 0 value expressions: expr: link_id type: bigint expr: a_lgc_eqp_id type: bigint expr: a_lgc_port_id type: bigint expr: z_lgc_eqp_id type: bigint expr: z_lgc_port_id type: bigint expr: local_area_id type: string expr: dt type: string Reduce Operator Tree: Join Operator condition map: Inner Join 0 to 1 condition expressions: 0 {VALUE._col1} {VALUE._col3} {VALUE._col4} {VALUE._col6} {VALUE._col7} {VALUE._col16} {VALUE._col17} 1 {VALUE._col0} {VALUE._col3} handleSkewJoin: false outputColumnNames: _col1, _col3, _col4, _col6, _col7, _col16, _col17, _col20, _col23 File Output Operator compressed: false GlobalTableId: 0 table: input format: org.apache.hadoop.mapred.SequenceFileInputFormat output format: org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat
Stage: Stage-2 Map Reduce Alias -> Map Operator Tree: $INTNAME Reduce Output Operator key expressions: expr: _col3 type: bigint sort order: + Map-reduce partition columns: expr: _col3 type: bigint tag: 0 value expressions: expr: _col20 type: bigint expr: _col23 type: int expr: _col1 type: bigint expr: _col3 type: bigint expr: _col4 type: bigint expr: _col6 type: bigint expr: _col7 type: bigint expr: _col16 type: string expr: _col17 type: string lea TableScan alias: lea Filter Operator predicate: expr: ((local_area_id = 20.0) and (dt = '20130801')) type: boolean Reduce Output Operator key expressions: expr: lgc_eqp_id type: bigint sort order: + Map-reduce partition columns: expr: lgc_eqp_id type: bigint tag: 2 value expressions: expr: res_spec_id type: int expr: local_area_id type: string expr: dt type: string lp2la TableScan alias: lp2la Filter Operator predicate: expr: ((local_area_id = 20.0) and (dt = '20130801')) type: boolean Reduce Output Operator key expressions: expr: lgc_eqp_id type: bigint sort order: + Map-reduce partition columns: expr: lgc_eqp_id type: bigint tag: 1 value expressions: expr: phy_eqp_id type: bigint expr: local_area_id type: string expr: dt type: string Reduce Operator Tree: Join Operator condition map: Inner Join 0 to 1 Inner Join 0 to 2 condition expressions: 0 {VALUE._col21} {VALUE._col23} {VALUE._col24} {VALUE._col26} {VALUE._col27} {VALUE._col36} {VALUE._col37} {VALUE._col0} {VALUE._col3} 1 {VALUE._col1} {VALUE._col8} {VALUE._col9} 2 {VALUE._col4} {VALUE._col19} {VALUE._col20} handleSkewJoin: false outputColumnNames: _col1, _col3, _col4, _col6, _col7, _col16, _col17, _col20, _col23, _col41, _col48, _col49, _col56, _col71, _col72 File Output Operator compressed: false GlobalTableId: 0 table: input format: org.apache.hadoop.mapred.SequenceFileInputFormat output format: org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat
Stage: Stage-3 Map Reduce Alias -> Map Operator Tree: $INTNAME Reduce Output Operator key expressions: expr: _col41 type: bigint sort order: + Map-reduce partition columns: expr: _col41 type: bigint tag: 0 value expressions: expr: _col20 type: bigint expr: _col23 type: int expr: _col1 type: bigint expr: _col3 type: bigint expr: _col4 type: bigint expr: _col6 type: bigint expr: _col7 type: bigint expr: _col16 type: string expr: _col17 type: string expr: _col56 type: int expr: _col71 type: string expr: _col72 type: string expr: _col41 type: bigint pea TableScan alias: pea Filter Operator predicate: expr: ((local_area_id = 20.0) and (dt = '20130801')) type: boolean Reduce Output Operator key expressions: expr: phy_eqp_id type: bigint sort order: + Map-reduce partition columns: expr: phy_eqp_id type: bigint tag: 1 value expressions: expr: res_spec_id type: int Reduce Operator Tree: Join Operator condition map: Inner Join 0 to 1 condition expressions: 0 {VALUE._col21} {VALUE._col23} {VALUE._col24} {VALUE._col26} {VALUE._col27} {VALUE._col36} {VALUE._col37} {VALUE._col0} {VALUE._col3} {VALUE._col64} {VALUE._col44} {VALUE._col59} {VALUE._col60} 1 {VALUE._col3} handleSkewJoin: false outputColumnNames: _col1, _col3, _col4, _col6, _col7, _col16, _col17, _col20, _col23, _col41, _col56, _col71, _col72, _col78 File Output Operator compressed: false GlobalTableId: 0 table: input format: org.apache.hadoop.mapred.SequenceFileInputFormat output format: org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat
Stage: Stage-4 Map Reduce Alias -> Map Operator Tree: $INTNAME Reduce Output Operator key expressions: expr: _col4 type: bigint sort order: + Map-reduce partition columns: expr: _col4 type: bigint tag: 0 value expressions: expr: _col20 type: bigint expr: _col23 type: int expr: _col1 type: bigint expr: _col3 type: bigint expr: _col4 type: bigint expr: _col6 type: bigint expr: _col7 type: bigint expr: _col16 type: string expr: _col17 type: string expr: _col78 type: int expr: _col56 type: int expr: _col71 type: string expr: _col72 type: string expr: _col41 type: bigint ll2pa TableScan alias: ll2pa Filter Operator predicate: expr: ((local_area_id = 20.0) and (dt = '20130801')) type: boolean Reduce Output Operator key expressions: expr: lgc_port_id type: bigint sort order: + Map-reduce partition columns: expr: lgc_port_id type: bigint tag: 1 value expressions: expr: phy_port_id type: bigint expr: local_area_id type: string expr: dt type: string lpa TableScan alias: lpa Filter Operator predicate: expr: ((local_area_id = 20.0) and (dt = '20130801')) type: boolean Reduce Output Operator key expressions: expr: lgc_port_id type: bigint sort order: + Map-reduce partition columns: expr: lgc_port_id type: bigint tag: 2 value expressions: expr: res_spec_id type: int expr: local_area_id type: string expr: dt type: string Reduce Operator Tree: Join Operator condition map: Inner Join 0 to 1 Inner Join 0 to 2 condition expressions: 0 {VALUE._col21} {VALUE._col23} {VALUE._col24} {VALUE._col26} {VALUE._col27} {VALUE._col36} {VALUE._col37} {VALUE._col0} {VALUE._col3} {VALUE._col43} {VALUE._col114} {VALUE._col94} {VALUE._col109} {VALUE._col110} 1 {VALUE._col1} {VALUE._col8} {VALUE._col9} 2 {VALUE._col4} {VALUE._col19} {VALUE._col20} handleSkewJoin: false outputColumnNames: _col1, _col3, _col4, _col6, _col7, _col16, _col17, _col20, _col23, _col43, _col91, _col106, _col121, _col122, _col126, _col133, _col134, _col141, _col156, _col157 File Output Operator compressed: false GlobalTableId: 0 table: input format: org.apache.hadoop.mapred.SequenceFileInputFormat output format: org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat
Stage: Stage-5 Map Reduce Alias -> Map Operator Tree: $INTNAME Reduce Output Operator key expressions: expr: _col126 type: bigint sort order: + Map-reduce partition columns: expr: _col126 type: bigint tag: 0 value expressions: expr: _col20 type: bigint expr: _col23 type: int expr: _col1 type: bigint expr: _col3 type: bigint expr: _col4 type: bigint expr: _col6 type: bigint expr: _col7 type: bigint expr: _col16 type: string expr: _col17 type: string expr: _col43 type: int expr: _col141 type: int expr: _col156 type: string expr: _col157 type: string expr: _col126 type: bigint expr: _col106 type: int expr: _col121 type: string expr: _col122 type: string expr: _col91 type: bigint ptpa TableScan alias: ptpa Filter Operator predicate: expr: ((local_area_id = 20.0) and (dt = '20130801')) type: boolean Reduce Output Operator key expressions: expr: phy_port_id type: bigint sort order: + Map-reduce partition columns: expr: phy_port_id type: bigint tag: 1 value expressions: expr: res_spec_id type: int Reduce Operator Tree: Join Operator condition map: Inner Join 0 to 1 condition expressions: 0 {VALUE._col21} {VALUE._col23} {VALUE._col24} {VALUE._col26} {VALUE._col27} {VALUE._col36} {VALUE._col37} {VALUE._col0} {VALUE._col3} {VALUE._col43} {VALUE._col94} {VALUE._col109} {VALUE._col110} {VALUE._col114} {VALUE._col149} {VALUE._col129} {VALUE._col144} {VALUE._col145} 1 {VALUE._col2} handleSkewJoin: false outputColumnNames: _col1, _col3, _col4, _col6, _col7, _col16, _col17, _col20, _col23, _col43, _col94, _col109, _col110, _col114, _col126, _col141, _col156, _col157, _col162 File Output Operator compressed: false GlobalTableId: 0 table: input format: org.apache.hadoop.mapred.SequenceFileInputFormat output format: org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat
Stage: Stage-6 Map Reduce Alias -> Map Operator Tree: $INTNAME Reduce Output Operator key expressions: expr: _col6 type: bigint sort order: + Map-reduce partition columns: expr: _col6 type: bigint tag: 0 value expressions: expr: _col20 type: bigint expr: _col23 type: int expr: _col1 type: bigint expr: _col3 type: bigint expr: _col4 type: bigint expr: _col6 type: bigint expr: _col7 type: bigint expr: _col16 type: string expr: _col17 type: string expr: _col162 type: int expr: _col43 type: int expr: _col94 type: int expr: _col109 type: string expr: _col110 type: string expr: _col114 type: bigint expr: _col141 type: int expr: _col126 type: bigint lez TableScan alias: lez Reduce Output Operator key expressions: expr: lgc_eqp_id type: bigint sort order: + Map-reduce partition columns: expr: lgc_eqp_id type: bigint tag: 2 value expressions: expr: res_spec_id type: int lp2lz TableScan alias: lp2lz Filter Operator predicate: expr: ((local_area_id = 20.0) and (dt = '20130801')) type: boolean Reduce Output Operator key expressions: expr: lgc_eqp_id type: bigint sort order: + Map-reduce partition columns: expr: lgc_eqp_id type: bigint tag: 1 value expressions: expr: phy_eqp_id type: bigint expr: local_area_id type: string expr: dt type: string Reduce Operator Tree: Join Operator condition map: Inner Join 0 to 1 Inner Join 0 to 2 condition expressions: 0 {VALUE._col21} {VALUE._col23} {VALUE._col24} {VALUE._col26} {VALUE._col27} {VALUE._col36} {VALUE._col37} {VALUE._col0} {VALUE._col3} {VALUE._col75} {VALUE._col42} {VALUE._col126} {VALUE._col141} {VALUE._col142} {VALUE._col146} {VALUE._col181} {VALUE._col161} 1 {VALUE._col1} {VALUE._col8} {VALUE._col9} 2 {VALUE._col4} handleSkewJoin: false outputColumnNames: _col1, _col3, _col4, _col6, _col7, _col16, _col17, _col20, _col23, _col43, _col92, _col126, _col141, _col142, _col146, _col158, _col173, _col193, _col200, _col201, _col208 File Output Operator compressed: false GlobalTableId: 0 table: input format: org.apache.hadoop.mapred.SequenceFileInputFormat output format: org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat
Stage: Stage-7 Map Reduce Alias -> Map Operator Tree: $INTNAME Reduce Output Operator key expressions: expr: _col193 type: bigint sort order: + Map-reduce partition columns: expr: _col193 type: bigint tag: 0 value expressions: expr: _col20 type: bigint expr: _col23 type: int expr: _col1 type: bigint expr: _col3 type: bigint expr: _col4 type: bigint expr: _col6 type: bigint expr: _col7 type: bigint expr: _col16 type: string expr: _col17 type: string expr: _col92 type: int expr: _col43 type: int expr: _col126 type: int expr: _col141 type: string expr: _col142 type: string expr: _col146 type: bigint expr: _col173 type: int expr: _col158 type: bigint expr: _col193 type: bigint expr: _col208 type: int pez TableScan alias: pez Filter Operator predicate: expr: ((local_area_id = 20.0) and (dt = '20130801')) type: boolean Reduce Output Operator key expressions: expr: phy_eqp_id type: bigint sort order: + Map-reduce partition columns: expr: phy_eqp_id type: bigint tag: 1 value expressions: expr: res_spec_id type: int Reduce Operator Tree: Join Operator condition map: Inner Join 0 to 1 condition expressions: 0 {VALUE._col21} {VALUE._col23} {VALUE._col24} {VALUE._col26} {VALUE._col27} {VALUE._col36} {VALUE._col37} {VALUE._col0} {VALUE._col3} {VALUE._col75} {VALUE._col42} {VALUE._col126} {VALUE._col141} {VALUE._col142} {VALUE._col146} {VALUE._col181} {VALUE._col161} {VALUE._col193} {VALUE._col208} 1 {VALUE._col3} handleSkewJoin: false outputColumnNames: _col1, _col3, _col4, _col6, _col7, _col16, _col17, _col20, _col23, _col43, _col92, _col126, _col141, _col142, _col146, _col158, _col173, _col193, _col208, _col230 File Output Operator compressed: false GlobalTableId: 0 table: input format: org.apache.hadoop.mapred.SequenceFileInputFormat output format: org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat
Stage: Stage-8 Map Reduce Alias -> Map Operator Tree: $INTNAME Reduce Output Operator key expressions: expr: _col7 type: bigint sort order: + Map-reduce partition columns: expr: _col7 type: bigint tag: 0 value expressions: expr: _col20 type: bigint expr: _col23 type: int expr: _col1 type: bigint expr: _col3 type: bigint expr: _col4 type: bigint expr: _col6 type: bigint expr: _col7 type: bigint expr: _col92 type: int expr: _col43 type: int expr: _col126 type: int expr: _col146 type: bigint expr: _col173 type: int expr: _col158 type: bigint expr: _col193 type: bigint expr: _col208 type: int expr: _col230 type: int ll2pz TableScan alias: ll2pz Filter Operator predicate: expr: ((local_area_id = 20.0) and (dt = '20130801')) type: boolean Reduce Output Operator key expressions: expr: lgc_port_id type: bigint sort order: + Map-reduce partition columns: expr: lgc_port_id type: bigint tag: 1 value expressions: expr: phy_port_id type: bigint expr: local_area_id type: string expr: dt type: string lpz TableScan alias: lpz Reduce Output Operator key expressions: expr: lgc_port_id type: bigint sort order: + Map-reduce partition columns: expr: lgc_port_id type: bigint tag: 2 value expressions: expr: res_spec_id type: int Reduce Operator Tree: Join Operator condition map: Inner Join 0 to 1 Inner Join 0 to 2 condition expressions: 0 {VALUE._col21} {VALUE._col23} {VALUE._col24} {VALUE._col26} {VALUE._col27} {VALUE._col0} {VALUE._col3} {VALUE._col75} {VALUE._col42} {VALUE._col126} {VALUE._col146} {VALUE._col181} {VALUE._col161} {VALUE._col193} {VALUE._col208} {VALUE._col230} 1 {VALUE._col1} {VALUE._col8} {VALUE._col9} 2 {VALUE._col4} handleSkewJoin: false outputColumnNames: _col1, _col3, _col4, _col6, _col7, _col20, _col23, _col43, _col92, _col126, _col146, _col158, _col173, _col193, _col208, _col230, _col278, _col285, _col286, _col293 File Output Operator compressed: false GlobalTableId: 0 table: input format: org.apache.hadoop.mapred.SequenceFileInputFormat output format: org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat
Stage: Stage-9 Map Reduce Alias -> Map Operator Tree: $INTNAME Reduce Output Operator key expressions: expr: _col278 type: bigint sort order: + Map-reduce partition columns: expr: _col278 type: bigint tag: 0 value expressions: expr: _col20 type: bigint expr: _col23 type: int expr: _col43 type: int expr: _col146 type: bigint expr: _col193 type: bigint expr: _col230 type: int expr: _col1 type: bigint expr: _col3 type: bigint expr: _col4 type: bigint expr: _col6 type: bigint expr: _col7 type: bigint expr: _col92 type: int expr: _col126 type: int expr: _col158 type: bigint expr: _col173 type: int expr: _col278 type: bigint expr: _col208 type: int expr: _col293 type: int ptpz TableScan alias: ptpz Filter Operator predicate: expr: ((local_area_id = 20.0) and (dt = '20130801')) type: boolean Reduce Output Operator key expressions: expr: phy_port_id type: bigint sort order: + Map-reduce partition columns: expr: phy_port_id type: bigint tag: 1 value expressions: expr: res_spec_id type: int Reduce Operator Tree: Join Operator condition map: Inner Join 0 to 1 condition expressions: 0 {VALUE._col0} {VALUE._col3} {VALUE._col23} {VALUE._col71} {VALUE._col83} {VALUE._col97} {VALUE._col145} {VALUE._col147} {VALUE._col148} {VALUE._col150} {VALUE._col151} {VALUE._col166} {VALUE._col200} {VALUE._col220} {VALUE._col235} {VALUE._col255} {VALUE._col270} {VALUE._col293} 1 {VALUE._col2} handleSkewJoin: false outputColumnNames: _col0, _col3, _col23, _col71, _col83, _col97, _col145, _col147, _col148, _col150, _col151, _col166, _col200, _col220, _col235, _col255, _col270, _col293, _col314 Select Operator expressions: expr: _col0 type: bigint expr: _col3 type: int expr: _col145 type: bigint expr: _col220 type: bigint expr: _col23 type: int expr: _col71 type: bigint expr: _col166 type: int expr: _col83 type: bigint expr: _col97 type: int expr: _col255 type: bigint expr: _col314 type: int expr: _col147 type: bigint expr: _col235 type: int expr: _col148 type: bigint expr: _col200 type: int expr: _col150 type: bigint expr: _col270 type: int expr: _col151 type: bigint expr: _col293 type: int outputColumnNames: _col0, _col1, _col2, _col3, _col4, _col5, _col6, _col7, _col8, _col9, _col10, _col11, _col12, _col13, _col14, _col15, _col16, _col17, _col18 Select Operator expressions: expr: _col0 type: bigint expr: UDFToLong(_col1) type: bigint expr: _col2 type: bigint expr: _col3 type: bigint expr: UDFToLong(_col4) type: bigint expr: _col5 type: bigint expr: UDFToLong(_col6) type: bigint expr: _col7 type: bigint expr: UDFToLong(_col8) type: bigint expr: _col9 type: bigint expr: UDFToLong(_col10) type: bigint expr: _col11 type: bigint expr: UDFToLong(_col12) type: bigint expr: _col13 type: bigint expr: UDFToLong(_col14) type: bigint expr: _col15 type: bigint expr: UDFToLong(_col16) type: bigint expr: _col17 type: bigint expr: UDFToLong(_col18) type: bigint outputColumnNames: _col0, _col1, _col2, _col3, _col4, _col5, _col6, _col7, _col8, _col9, _col10, _col11, _col12, _col13, _col14, _col15, _col16, _col17, _col18 File Output Operator compressed: false GlobalTableId: 1 table: input format: org.apache.hadoop.mapred.TextInputFormat output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe name: default.mid_res_link_lgc_bt
Stage: Stage-0 Move Operator tables: partition: dt 20130831 local_area_id 20 replace: true table: input format: org.apache.hadoop.mapred.TextInputFormat output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe name: default.mid_res_link_lgc_bt
Stage: Stage-10 Stats-Aggr Operator |
查看任务执行，有Map数2个，1个reduce。错误发生在map阶段，两个map阶段，错误后重新执行4次，均失败。
进入查看详细的每次task运行出错日志
|
2013-09-11 09:42:44,392 WARN org.apache.hadoop.util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable 2013-09-11 09:42:44,774 WARN mapreduce.Counters: Group org.apache.hadoop.mapred.Task$Counter is deprecated. Use org.apache.hadoop.mapreduce.TaskCounter instead 2013-09-11 09:42:45,257 INFO org.apache.hadoop.mapred.TaskRunner: Creating symlink: /app/tmp/hdfs/tmp/mapred/local/taskTracker/distcache/-1273717043127285981_-158808503_179256813/tascluster/tmp/hive-hadoop/hive_2013-09-11_09-42-32_079_3962951880312773437/-mr-10011/4659e5b9-e5ba-4a26-9a81-c82fbdbf0af3 <- /app/tmp/hdfs/tmp/mapred/local/taskTracker/hadoop/jobcache/job_201309102223_0001/attempt_201309102223_0001_m_000000_0/work/HIVE_PLAN4659e5b9-e5ba-4a26-9a81-c82fbdbf0af3 2013-09-11 09:42:45,279 INFO org.apache.hadoop.filecache.TrackerDistributedCacheManager: Creating symlink: /app/tmp/hdfs/tmp/mapred/local/taskTracker/hadoop/jobcache/job_201309102223_0001/jars/.job.jar.crc <- /app/tmp/hdfs/tmp/mapred/local/taskTracker/hadoop/jobcache/job_201309102223_0001/attempt_201309102223_0001_m_000000_0/work/.job.jar.crc 2013-09-11 09:42:45,292 INFO org.apache.hadoop.filecache.TrackerDistributedCacheManager: Creating symlink: /app/tmp/hdfs/tmp/mapred/local/taskTracker/hadoop/jobcache/job_201309102223_0001/jars/job.jar <- /app/tmp/hdfs/tmp/mapred/local/taskTracker/hadoop/jobcache/job_201309102223_0001/attempt_201309102223_0001_m_000000_0/work/job.jar 2013-09-11 09:42:45,362 WARN org.apache.hadoop.conf.Configuration: session.id is deprecated. Instead, use dfs.metrics.session-id 2013-09-11 09:42:45,364 INFO org.apache.hadoop.metrics.jvm.JvmMetrics: Initializing JVM Metrics with processName=MAP, sessionId= 2013-09-11 09:42:45,731 INFO org.apache.hadoop.util.ProcessTree: setsid exited with exit code 0 2013-09-11 09:42:45,760 INFO org.apache.hadoop.mapred.Task: Using ResourceCalculatorPlugin : org.apache.hadoop.util.LinuxResourceCalculatorPlugin@5b32627 2013-09-11 09:42:46,050 INFO org.apache.hadoop.mapred.MapTask: Processing split: Paths:/tmp/hive-hadoop/hive_2013-09-11_09-42-32_079_3962951880312773437/-mr-10010/1/emptyFile:0+0InputFormatClass: org.apache.hadoop.mapred.TextInputFormat
2013-09-11 09:42:46,087 WARN org.apache.hadoop.hive.conf.HiveConf: hive-site.xml not found on CLASSPATH 2013-09-11 09:42:46,371 INFO org.apache.hadoop.hive.ql.io.HiveContextAwareRecordReader: Processing file hdfs://tascluster/tmp/hive-hadoop/hive_2013-09-11_09-42-32_079_3962951880312773437/-mr-10010/1/emptyFile 2013-09-11 09:42:46,372 WARN mapreduce.Counters: Counter name MAP_INPUT_BYTES is deprecated. Use FileInputFormatCounters as group name and BYTES_READ as counter name instead 2013-09-11 09:42:46,375 INFO org.apache.hadoop.mapred.MapTask: numReduceTasks: 1 2013-09-11 09:42:46,380 INFO org.apache.hadoop.mapred.MapTask: Map output collector class = org.apache.hadoop.mapred.MapTask$MapOutputBuffer 2013-09-11 09:42:46,383 INFO org.apache.hadoop.mapred.MapTask: io.sort.mb = 100 2013-09-11 09:42:46,642 INFO org.apache.hadoop.mapred.TaskLogsTruncater: Initializing logs' truncater with mapRetainSize=-1 and reduceRetainSize=-1 2013-09-11 09:42:46,678 FATAL org.apache.hadoop.mapred.Child: Error running child : java.lang.OutOfMemoryError: Java heap space at org.apache.hadoop.mapred.MapTask$MapOutputBuffer.init(MapTask.java:827) at org.apache.hadoop.mapred.MapTask.createSortingCollector(MapTask.java:377) at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:407) at org.apache.hadoop.mapred.MapTask.run(MapTask.java:333) at org.apache.hadoop.mapred.Child$4.run(Child.java:268) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:416) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1408) at org.apache.hadoop.mapred.Child.main(Child.java:262)
|
看到最后的错误日志应该是mapred.Child的内存太小了，所以加大task的内存限制。
修改 mapred.child.java.opts 参数，原来设置的是100M，该为200。问题解决
