Spark源码系列03-任务提交02

spark源码系列03-任务提交01：https://blog.csdn.net/qq_19968255/article/details/82803794

 

12. DAGScheduler    getShuffleMapStage

创建父Stage

val stage =newOrUsedStage(……

 

13. DAGScheduler    newOrUsedStage

递归

val stage = newStage(rdd, numTasks, Some(shuffleDep), jobId, callSite)

 

14．DAGScheduler    handleJobSubmitted

……

if (shouldRunLocally) {

        // Compute very short actions like first() or take() with no parent stages locally.

        //TODO 本地运行

        listenerBus.post(

          SparkListenerJobStart(job.jobId, jobSubmissionTime, Seq.empty, properties))

        runLocally(job)

      } else {

        //TODO 集群模式