dremio 的 YarnDaemon 简单说明

YarnDaemon 与DremioDaemon 基本类似，都是启动dremio 服务，只是YarnDaemon 更多是关于执行器节点的运行模式与
DremioDaemon 差异比较大，DremioDaemon 就是一个long running 任务，但是YarnDaemon 实际上是一个由线程调度出发的
任务，运行时间不如DremioDaemon长，同时会有watchdog 进行服务状态的监控

参考代码

 
public void run() {
  try (TimedBlock b = Timer.time("main")) {
 
    // create a temporary local write path
    final Path localWritePath = Files.createTempDirectory("dremio-executor");
    final DACConfig config = DACConfig.newConfig()
        .writePath(localWritePath.toString());
    logger.info("Local write path set to '{}'", localWritePath);
 
    final SabotConfig sabotConfig = config.getConfig().getSabotConfig();
    final DACModule module = sabotConfig.getInstance(DremioDaemon.DAEMON_MODULE_CLASS, DACModule.class, DACDaemonModule.class);
    try (final DACDaemon daemon = DACDaemon.newDremioDaemon(config, ClassPathScanner.fromPrescan(sabotConfig),
        module)) {
      dacDaemon = daemon;
      // 此部分属于标准DACDaemon 玩法
      daemon.init();
      // Start yarn watchdog
      // 特定于yarn 模式的watchdog
      startYarnWatchdog(daemon);
      daemon.closeOnJVMShutDown();
      daemon.awaitClose();
    }
  } catch (Exception e) {
    throw Throwables.propagate(e);
  }
}

YarnDaemon 的启动

YarnDaemon 是直接由YarnController使用的，属于标准的apache twill

// 启动集群
public TwillController startCluster(YarnConfiguration yarnConfiguration, List<Property> propertyList) {
    TwillController tmpController = createPreparer(yarnConfiguration, propertyList).start();
    return tmpController;
  }
 // TwillPreparer 包装
  protected TwillPreparer createPreparer(YarnConfiguration yarnConfiguration, List<Property> propertyList) {
    // 入口参数定义，就使用了YarnDaemon
    AppBundleRunnable.Arguments discoveryArgs = new AppBundleRunnable.Arguments(
        YARN_BUNDLED_JAR_NAME,
        "com.dremio.dac.daemon.YarnDaemon",
        new String[] {});
 
    DacDaemonYarnApplication dacDaemonApp = new DacDaemonYarnApplication(dremioConfig, yarnConfiguration,
      new DacDaemonYarnApplication.Environment());
 
    TwillRunnerService twillRunner = startTwillRunner(yarnConfiguration);
 
    Map<String, String> envVars = Maps.newHashMap();
    envVars.put("MALLOC_ARENA_MAX", "4");
    envVars.put("MALLOC_MMAP_THRESHOLD_", "131072");
    envVars.put("MALLOC_TRIM_THRESHOLD_", "131072");
    envVars.put("MALLOC_TOP_PAD_", "131072");
    envVars.put("MALLOC_MMAP_MAX_", "65536");
    // Set ${DREMIO_HOME} for YarnDaemon to avoid config substitution failure
    envVars.put(DREMIO_HOME, ".");
 
    try {
      String userName = UserGroupInformation.getCurrentUser().getUserName();
      envVars.put("HADOOP_USER_NAME", userName);
    } catch (IOException e) {
      logger.error("Exception while trying to fill out HADOOP_USER_NAME with current user", e);
    }
 
    for (Property prop : propertyList) {
      // add if it is env var
      if (PropertyType.ENV_VAR.equals(prop.getType())) {
        envVars.put(prop.getKey(), prop.getValue());
      }
    }
    String[] yarnClasspath = yarnConfiguration.getStrings(YarnConfiguration.YARN_APPLICATION_CLASSPATH,
      YarnConfiguration.DEFAULT_YARN_APPLICATION_CLASSPATH);
    final TwillPreparer preparer = twillRunner.prepare(dacDaemonApp)
      .addLogHandler(new YarnTwillLogHandler())
      .withApplicationClassPaths(yarnClasspath)
      .withBundlerClassAcceptor(new HadoopClassExcluder())
      .setLogLevels(ImmutableMap.of(Logger.ROOT_LOGGER_NAME, yarnContainerLogLevel()))
      .withEnv(YARN_RUNNABLE_NAME, envVars)
      .withMaxRetries(YARN_RUNNABLE_NAME, MAX_APP_RESTART_RETRIES)
      .withArguments(YARN_RUNNABLE_NAME, discoveryArgs.toArray())
      .setJVMOptions(YARN_RUNNABLE_NAME, prepareCommandOptions(yarnConfiguration, propertyList))
      .withClassPaths(dacDaemonApp.getJarNames());
 
    String queue = yarnConfiguration.get(DacDaemonYarnApplication.YARN_QUEUE_NAME);
    if (queue != null) {
      preparer.setSchedulerQueue(queue);
    }
 
    if (dremioConfig.getBoolean(DremioConfig.DEBUG_YARN_ENABLED)) {
      preparer.enableDebugging(true, YARN_RUNNABLE_NAME);
    }
 
    return preparer;
  }

说明

以上只是一个简单的说明，后变会介绍下关于yarn 包装调度部分，通过以上我们可以大致了解到是如果关联系统的

参考资料

dac/daemon/src/main/java/com/dremio/dac/daemon/YarnDaemon.java
provision/yarn/yarntwill/src/main/java/com/dremio/provision/yarn/YarnController.java

posted on 2023-02-04 13:08  荣锋亮  阅读(28)  评论(0)  编辑  收藏  举报

导航