TaskTracker与JobTracker的通信

　　JobTracker与TaskTracker之间采用了pull通信模型，即JobTracker从不会主动与TaskTracker通信，而总是被动等待TaskTracker汇报信息并领取其对应的命令。他们之间通过InterTrackerProtocol协议进行通信。

　　TaskTracker的main方法如下

public static void main(String argv[]) throws Exception {
        ......
        TaskTracker tt = new TaskTracker(conf);
        MBeans.register("TaskTracker", "TaskTrackerInfo", tt);
        tt.run(); //启动TaskTracker的run方法
        ......
    }

　　在run方法中 State osState = offerService() 这一行代码开始启动TaskTracker的各种服务。offerService()中通过HeartbeatResponse heartbeatResponse = transmitHeartBeat(now) 调用了transmitHeartBeat方法进行心跳连接，关键代码如下：

HeartbeatResponse heartbeatResponse = jobClient.heartbeat(status, justStarted, justInited, askForNewTask, heartbeatResponseId);

　　返回JobTracker类，在TaskTracker中调用了它的heartbeat方法，如下

public synchronized HeartbeatResponse heartbeat(TaskTrackerStatus status, boolean restarted, boolean initialContact, boolean acceptNewTasks, short responseId) throws IOException {
        ......
        HeartbeatResponse response = new HeartbeatResponse(newResponseId, null);
        List<TaskTrackerAction> actions = new ArrayList<TaskTrackerAction>();
        boolean isBlacklisted = faultyTrackers.isBlacklisted(status.getHost());
        
        if (recoveryManager.shouldSchedule() && acceptNewTasks && !isBlacklisted) {
            TaskTrackerStatus taskTrackerStatus = getTaskTrackerStatus(trackerName);
            if (taskTrackerStatus == null) {
                LOG.warn("Unknown task tracker polling; ignoring: " + trackerName);
            } else {
                //此处开始调用 mapTask 和 reduceTask
                List<Task> tasks = getSetupAndCleanupTasks(taskTrackerStatus);
                if (tasks == null) {
                    //为TaskTracker分配任务
                    tasks = taskScheduler.assignTasks(taskTrackers.get(trackerName));
                }
                if (tasks != null) {
                    for (Task task : tasks) {
                        expireLaunchingTasks.addNewTask(task.getTaskID());
                        if (LOG.isDebugEnabled()) {
                            LOG.debug(trackerName + " -> LaunchTask: " + task.getTaskID());
                        }
                        //启动新任务
                        actions.add(new LaunchTaskAction(task));
                    }
                }
            }
        }

        // 杀死任务
        List<TaskTrackerAction> killTasksList = getTasksToKill(trackerName);
        if (killTasksList != null) {
            actions.addAll(killTasksList);
        }

        // 杀死作业
        List<TaskTrackerAction> killJobsList = getJobsForCleanup(trackerName);
        if (killJobsList != null) {
            actions.addAll(killJobsList);
        }

        // 提交任务
        List<TaskTrackerAction> commitTasksList = getTasksToSave(status);
        if (commitTasksList != null) {
            actions.addAll(commitTasksList);
        }

        // calculate next heartbeat interval and put in heartbeat response
        int nextInterval = getNextHeartbeatInterval();
        response.setHeartbeatInterval(nextInterval);
        response.setActions(actions.toArray(new TaskTrackerAction[actions.size()]));

        // check if the restart info is req
        if (addRestartInfo) {
            response.setRecoveredJobs(recoveryManager.getJobsToRecover());
        }

        // Update the trackerToHeartbeatResponseMap
        trackerToHeartbeatResponseMap.put(trackerName, response);

        // Done processing the hearbeat, now remove 'marked' tasks
        removeMarkedTasks(trackerName);

        return response;
    }

 　　下面看一下 getSetupAndCleanupTasks()方法，该方法中执行 job.obtainTaskCleanupTask(taskTracker, true) 来获得Task任务。

在obtainTaskCleanupTask(taskTracker, true)方法中执行tip.addRunningTask(taskid, taskTracker, true)来添加运行的任务。

public Task addRunningTask(TaskAttemptID taskid, String taskTracker, boolean taskCleanup) {
        // 1 slot is enough for taskCleanup task
        int numSlotsNeeded = taskCleanup ? 1 : numSlotsRequired;
        // create the task
        Task t = null;
        if (isMapTask()) {
            if (LOG.isDebugEnabled()) {
                LOG.debug("attempt " + numTaskFailures + " sending skippedRecords " + failedRanges.getIndicesCount());
            }
            t = new MapTask(jobFile, taskid, partition, splitInfo.getSplitIndex(), numSlotsNeeded);
        } else {
            t = new ReduceTask(jobFile, taskid, partition, numMaps, numSlotsNeeded);
        }
　　　　　......return t;
    }

　　该方法开始确定是MapTask还是ReduceTask。

　　在MapTask或ReduceTask的run方法里都进行了是否执行新api的判断，如MapTask中

 public void run(final JobConf job, final TaskUmbilicalProtocol umbilical) 
    throws IOException, ClassNotFoundException, InterruptedException {

    if (useNewApi) {
      runNewMapper(job, splitMetaInfo, umbilical, reporter);
    } else {
      runOldMapper(job, splitMetaInfo, umbilical, reporter);
    }
    done(umbilical, reporter);
  }

在runNewMapper 或 runOldMapper 方法里通过反射的方式ReflectionUtils.newInstance实例化在job里设置的mapClass。