`

yarn-similar logs when starting up container

 
阅读更多
15/12/09 16:47:52 INFO yarn.ExecutorRunnable: Setting up executor with environment: Map(CLASSPATH -> {{PWD}}<CPS>{{PWD}}/__spark__.jar<CPS>$HADOOP_CONF_DIR<CPS>$HADOOP_COMMON_HOME/s
hare/hadoop/common/*<CPS>$HADOOP_COMMON_HOME/share/hadoop/common/lib/*<CPS>$HADOOP_HDFS_HOME/share/hadoop/hdfs/*<CPS>$HADOOP_HDFS_HOME/share/hadoop/hdfs/lib/*<CPS>$HADOOP_YARN_HOME/
share/hadoop/yarn/*<CPS>$HADOOP_YARN_HOME/share/hadoop/yarn/lib/*<CPS>$HADOOP_MAPRED_HOME/share/hadoop/mapreduce/*<CPS>$HADOOP_MAPRED_HOME/share/hadoop/mapreduce/lib/*, SPARK_LOG_UR
L_STDERR -> http://gzsw-11:8042/node/containerlogs/container_1441038159113_0018_01_000003/hadoop/stderr?start=0, SPARK_YARN_STAGING_DIR -> .sparkStaging/application_1441038159113_00
18, SPARK_YARN_CACHE_FILES_FILE_SIZES -> 157585209, SPARK_USER -> hadoop, SPARK_YARN_CACHE_FILES_VISIBILITIES -> PRIVATE, SPARK_YARN_MODE -> true, SPARK_YARN_CACHE_FILES_TIME_STAMPS
 -> 1449650858015, SPARK_LOG_URL_STDOUT -> http://gzsw-11:8042/node/containerlogs/container_1441038159113_0018_01_000003/hadoop/stdout?start=0, SPARK_YARN_CACHE_FILES -> hdfs://mycl
uster/user/hadoop/.sparkStaging/application_1441038159113_0018/spark-assembly-1.4.1-hadoop2.4.0.jar#__spark__.jar)

15/12/09 16:47:52 INFO yarn.ExecutorRunnable: Setting up executor with commands: List({{JAVA_HOME}}/bin/java, -server, -XX:OnOutOfMemoryError='kill %p', -Xms2048m, -Xmx2048m, '-Xlog
gc:~/spark-executor.gc', '-XX:+UseCMSCompactAtFullCollection', '-XX:CMSFullGCsBeforeCompaction=2', '-XX:CMSInitiatingOccupancyFraction=65', '-XX:+UseCMSInitiatingOccupancyOnly', '-X
X:PermSize=64m', '-XX:MaxPermSize=256m', '-XX:NewRatio=5', '-XX:+UseParNewGC', '-XX:+UseConcMarkSweepGC', '-XX:+PrintGCDateStamps', '-XX:+PrintGCDetails', '-XX:ParallelGCThreads=5',
 -Djava.io.tmpdir={{PWD}}/tmp, '-Dspark.master.ui.port=7102', '-Dspark.worker.ui.port=7105', '-Dspark.ui.port=7106', '-Dspark.driver.port=56894', -Dspark.yarn.app.container.log.dir=
<LOG_DIR>, org.apache.spark.executor.CoarseGrainedExecutorBackend, --driver-url, akka.tcp://sparkDriver@192.168.100.4:56894/user/CoarseGrainedScheduler, --executor-id, 2, --hostname
, gzsw-11, --cores, 2, --app-id, application_1441038159113_0018, --user-class-path, file:$PWD/__app__.jar, 1>, <LOG_DIR>/stdout, 2>, <LOG_DIR>/stderr)

   by one more checking ,that's one launched command for executor,and here one container will startup as executor says CoarseGrainedExecutorBackend,ie same jvm with container instead of spawning up new jvm process.

0
0
分享到:
评论

相关推荐

    hadoop-yarn-client-2.6.5-API文档-中文版.zip

    赠送jar包:hadoop-yarn-client-2.6.5.jar; 赠送原API文档:hadoop-yarn-client-2.6.5-javadoc.jar; 赠送源代码:hadoop-yarn-client-2.6.5-sources.jar; 赠送Maven依赖信息文件:hadoop-yarn-client-2.6.5.pom;...

    hadoop-yarn-common-2.6.5-API文档-中文版.zip

    赠送jar包:hadoop-yarn-common-2.6.5.jar 赠送原API文档:hadoop-yarn-common-2.6.5-javadoc.jar 赠送源代码:hadoop-yarn-common-2.6.5-sources.jar 包含翻译后的API文档:hadoop-yarn-common-2.6.5-javadoc-...

    hadoop-yarn-server-resourcemanager-2.6.0-API文档-中文版.zip

    赠送jar包:hadoop-yarn-server-resourcemanager-2.6.0.jar; 赠送原API文档:hadoop-yarn-server-resourcemanager-2.6.0-javadoc.jar; 赠送源代码:hadoop-yarn-server-resourcemanager-2.6.0-sources.jar; 赠送...

    hadoop-yarn-api-2.5.1-API文档-中文版.zip

    赠送jar包:hadoop-yarn-api-2.5.1.jar; 赠送原API文档:hadoop-yarn-api-2.5.1-javadoc.jar; 赠送源代码:hadoop-yarn-api-2.5.1-sources.jar; 赠送Maven依赖信息文件:hadoop-yarn-api-2.5.1.pom; 包含翻译后...

    yarn-v1.22.5.tar.gz

    1. 下载并解压:首先从源下载 `yarn-v1.22.5.tar.gz` 文件,然后使用 `tar -zxvf yarn-v1.22.5.tar.gz` 命令进行解压。 2. 安装:解压后,进入解压后的目录,通过运行 `./install.sh` 或者 `sudo ./install.sh`...

    hadoop-yarn-api-2.7.3-API文档-中英对照版.zip

    赠送jar包:hadoop-yarn-api-2.7.3.jar; 赠送原API文档:hadoop-yarn-api-2.7.3-javadoc.jar; 赠送源代码:hadoop-yarn-api-2.7.3-sources.jar; 赠送Maven依赖信息文件:hadoop-yarn-api-2.7.3.pom; 包含翻译后...

    hadoop-yarn-server-common-2.6.5-API文档-中文版.zip

    赠送jar包:hadoop-yarn-server-common-2.6.5.jar; 赠送原API文档:hadoop-yarn-server-common-2.6.5-javadoc.jar; 赠送源代码:hadoop-yarn-server-common-2.6.5-sources.jar; 赠送Maven依赖信息文件:hadoop-...

    spark-2.2.0-yarn-shuffle.jar

    spark-2.2.0-yarn-shuffle.jar

    yarn-1.22.10编译工具

    yarn-1.22.10编译工具yarn-1.22.10编译工具 yarn-1.22.10编译工具yarn-1.22.10编译工具 yarn-1.22.10编译工具yarn-1.22.10编译工具 yarn-1.22.10编译工具yarn-1.22.10编译工具 yarn-1.22.10编译工具yarn-1.22.10编译...

    hadoop-yarn-common-2.5.1-API文档-中英对照版.zip

    赠送jar包:hadoop-yarn-common-2.5.1.jar; 赠送原API文档:hadoop-yarn-common-2.5.1-javadoc.jar; 赠送源代码:hadoop-yarn-common-2.5.1-sources.jar; 包含翻译后的API文档:hadoop-yarn-common-2.5.1-...

    SPARK2_ON_YARN-2.4.0.cloudera2.jar

    SPARK2_ON_YARN-2.4.0 jar包下载

    yarn-v0.23.2.tar.gz

    yarn-v0.23.2.tar.gz 在安装ambari,源码编译的时候下载的文件有问题 手动下载 地址 https://github.com/yarnpkg/yarn/releases/download/v0.23.2/yarn-v0.23.2.tar.gz

    前端开源库-yarn-deduplicate

    `yarn-deduplicate`就是这样一个工具,它专为解决`yarn.lock`文件中的依赖重复问题而设计,从而帮助开发者更好地管理和优化项目依赖。本文将详细讲解`yarn-deduplicate`的工作原理、使用方法以及其在前端开源生态...

    yarn-1.22.0.msi

    现在官网下载缓慢,这里可以直接下载最新版本安装,版本号 yarn-1.22.0.msi yarn是facebook发布的一款取代npm的包管理工具。 快速、可靠、安全的依赖管理工具

    前端开源库-yarn-update-lock

    "yarn-update-lock"是一个针对前端开源库的实用工具,专注于管理和优化`yarn.lock`文件。这个工具的目的是确保项目依赖的版本是最新的,并且消除其中可能存在的重复项,从而保持项目的整洁和稳定。 `yarn.lock`文件...

    yarn-utils.py

    计算yarn内存配置的python脚本yarn-util.py,此脚本有四个参数 参数 描述 -c CORES 每个节点CPU核数 -m MEMORY 每个节点内存总数(单位G) -d DISKS 每个节点的硬盘个数 -k HBASE 如果安装了Hbase则为True,否则为...

    hadoop-yarn-server-web-proxy-2.6.0-API文档-中文版.zip

    赠送jar包:hadoop-yarn-server-web-proxy-2.6.0.jar; 赠送原API文档:hadoop-yarn-server-web-proxy-2.6.0-javadoc.jar; 赠送源代码:hadoop-yarn-server-web-proxy-2.6.0-sources.jar; 赠送Maven依赖信息文件:...

    hadoop-yarn-server-resourcemanager-2.7.4.jar

    hadoop2.7.4安装包补丁包,解决yarn定时调度启动问题!!

    spark-1.6.1-yarn-shuffle.jar

    spark-1.6.1-yarn-shuffle.jar 下载。spark-1.6.1-yarn-shuffle.jar 下载。spark-1.6.1-yarn-shuffle.jar 下载。

    yarn-1.22.4.msi和yarn-1.22.5.msi

    这两个文件,"yarn-1.22.4.msi" 和 "yarn-1.22.5.msi",是Yarn的特定版本安装程序,适用于Windows操作系统,因为它们的扩展名是".msi",这是Microsoft Installer的文件格式。 Yarn 1.22.4和1.22.5是稳定版本,每个...

Global site tag (gtag.js) - Google Analytics