随笔分类 -  hadoop

摘要:/** Splits files returned by {@link #listStatus(JobConf)} when * they're too big.*/ public InputSplit[] getSplits(JobConf job, int numSplits) throws IOException { //计时器, StopWatch ... 阅读全文
posted @ 2018-07-21 15:17 天添 阅读(615) 评论(0) 推荐(0)
摘要:/** * * @param job 配置信息 * @param split split * @param recordDelimiter 分列的字符 * @throws IOException */ public LineRecordReader(Configuration job, FileSplit split, byte[] r... 阅读全文
posted @ 2018-07-21 13:02 天添 阅读(728) 评论(0) 推荐(0)
摘要:/** A section of an input file. Returned by {@link * InputFormat#getSplits(JobContext)} and passed to * {@link InputFormat#createRecordReader(InputSplit,TaskAttemptContext)}. * * 文件的一部分,通过InputForm... 阅读全文
posted @ 2018-07-20 10:49 天添 阅读(982) 评论(0) 推荐(0)
摘要:/** * InputSplit represents the data to be processed by an * individual {@link Mapper}. * InputSplit 代表可以被Mapper处理的数据 * Typically, it presents a byte-oriented view on the input and is the * r... 阅读全文
posted @ 2018-07-20 10:40 天添 阅读(661) 评论(0) 推荐(0)