spark-学习笔记--25 executor

Executor

 

 

1、反序列化 task  交给 executor 执行

 

 

 

2、org.apache.spark.executor.Executor 的方法：

 

def launchTask(context: ExecutorBackend,taskId: 
               Long,attemptNumber: Int,taskName: String,
	       serializedTask: ByteBuffer): Unit = {
    val tr = new TaskRunner(context, taskId = taskId, attemptNumber = attemptNumber,
                            taskName,serializedTask)
    runningTasks.put(taskId, tr)
    threadPool.execute(tr)
}

 

   executor  launchTask 方法对每一个 task创建一个 TaskRunner 

   TaskRunner  extends Runnable：

    

   class TaskRunner(execBackend: ExecutorBackend,val taskId: Long,
                    val attemptNumber: Int,taskName: String,
                    serializedTask: ByteBuffer)
    extends Runnable { ...}

 

  其实就是将task 封装到线程任务(TaskRunner) 放入线程池执行 

 

 

 

3、 图：