`

Hadoop启动时出现Unrecognized option: -jvm 和 Could not create the Java virtual machine

 
阅读更多

http://javoft.net/2011/06/hadoop-unrecognized-option-jvm-could-not-create-the-java-virtual-machine/

 

根据官方文档配置单节点的时候,在运行sh start-all.sh时遇到出现问题:


localhost: starting datanode, logging to /usr/local/hadoop/bin/../logs/hadoop-root-datanode-fedora.out
localhost: Unrecognized option: -jvm
localhost: Could not create the Java virtual machine.

在网上搜到很多 Could not create the Java virtual machine.都是说内存分配不够或者其他的问题,跟Hadoop相关的不多。搜了很多最后找到了这个问题的原因,首先说一下我安装过程。

因为是本机,所以为了方便加上习惯性su开启root运行,故同样也用root来运行start-all.sh,于是上面问题出现了,到网上搜了,有人说改/hadoop/conf/hadoop-env.sh中的


# The maximum amount of heap to use, in MB. Default is 1000.
# export HADOOP_HEAPSIZE=2000

把 HADOOP_HEAPSIZE的值设为一个更小的,当然这样说的人肯定是以为jvm不能启动是因为内存大小没满足1000的要求,内存不足使得 Could not create the Java virtual machine,看起来貌似有道理,但是怎么解释-jvm的问题呢?动手改改看,改过以后,无果,至少说明我遇到的不是这问题。继续搜,最后发现是在hadoop/bin/hadoop中有如下一段shell:

1
2
3
4
5
6
CLASS='org.apache.hadoop.hdfs.server.datanode.DataNode'
if [[ $EUID -eq 0 ]]; then
HADOOP_OPTS="$HADOOP_OPTS -jvm server $HADOOP_DATANODE_OPTS"
else
HADOOP_OPTS="$HADOOP_OPTS -server $HADOOP_DATANODE_OPTS"
fi

 

其中的

如果 $EUID 为 0,什么意思呢?
有效用户标识号(EUID):该标识号负责标识以什么用户身份来给新创建的进程赋所有权、检查文件的存取权限和检查通过系统调用kill向进程发送软 中断信号的许可权限。
在root用户下echo $EUID,echo结果为 0
ok,在root下会有-jvm选项添加上去,上面说的Unrecognized option: -jvm难道就是这里产生的。

1
2
if [[ $EUID -eq 0 ]]; then
HADOOP_OPTS="$HADOOP_OPTS -jvm server $HADOOP_DATANODE_OPTS"

 

两个想法。一个想法是自己改了这shell代码,去掉里面的-jvm。另外一个想法是 既然这里要求 $EUID -eq 0,那别用$EUID不为0的(root用户)用户运行即可。果断试试,换上普通用户根据文档提示做。ok,成功。好奇的再试试第一个想法,其实暂时还是不太想动源码。但是这shell动动也没妨,果断去掉上面的-jvm,直接把上面的if else 结构直接去掉改为

1
HADOOP_OPTS="$HADOOP_OPTS -server $HADOOP_DATANODE_OPTS"

同样运行成功。

如上几个尝试最终证明问题就出现在与-jvm这个参数选项,这是由于Hadoop版本或者java版本差异产生的问题吗?继续研究中…

分享到:
评论
1 楼 xunke515 2013-08-20  
请问如何更换其他用户,注销了从新登录的时候选择么?

相关推荐

    hadoop配置运行错误

    问题描述:在hadoop安装目录/bin/hadoop中有一段shell代码导致Hadoop启动datanode时出现Unrecognized option: -jvm和Could not create the Java virtual machine错误。 解决方法: 1. 修改hadoop安装目录/bin/...

    hadoop-mapreduce-client-common-2.6.5-API文档-中英对照版.zip

    赠送jar包:hadoop-mapreduce-client-common-2.6.5.jar; 赠送原API文档:hadoop-mapreduce-client-common-2.6.5-javadoc.jar; 赠送源代码:hadoop-mapreduce-client-common-2.6.5-sources.jar; 赠送Maven依赖信息...

    hadoop-mapreduce-client-jobclient-2.6.5-API文档-中文版.zip

    赠送jar包:hadoop-mapreduce-client-jobclient-2.6.5.jar; 赠送原API文档:hadoop-mapreduce-client-jobclient-2.6.5-javadoc.jar; 赠送源代码:hadoop-mapreduce-client-jobclient-2.6.5-sources.jar; 赠送...

    hadoop-mapreduce-client-core-2.5.1-API文档-中文版.zip

    赠送jar包:hadoop-mapreduce-client-core-2.5.1.jar; 赠送原API文档:hadoop-mapreduce-client-core-2.5.1-javadoc.jar; 赠送源代码:hadoop-mapreduce-client-core-2.5.1-sources.jar; 赠送Maven依赖信息文件:...

    hadoop-yarn-client-2.6.5-API文档-中文版.zip

    赠送jar包:hadoop-yarn-client-2.6.5.jar; 赠送原API文档:hadoop-yarn-client-2.6.5-javadoc.jar; 赠送源代码:hadoop-yarn-client-2.6.5-sources.jar; 赠送Maven依赖信息文件:hadoop-yarn-client-2.6.5.pom;...

    hadoop-yarn-api-2.5.1-API文档-中文版.zip

    赠送jar包:hadoop-yarn-api-2.5.1.jar; 赠送原API文档:hadoop-yarn-api-2.5.1-javadoc.jar; 赠送源代码:hadoop-yarn-api-2.5.1-sources.jar; 赠送Maven依赖信息文件:hadoop-yarn-api-2.5.1.pom; 包含翻译后...

    hadoop-mapreduce-client-app-2.7.3-API文档-中英对照版.zip

    赠送jar包:hadoop-mapreduce-client-app-2.7.3.jar; 赠送原API文档:hadoop-mapreduce-client-app-2.7.3-javadoc.jar; 赠送源代码:hadoop-mapreduce-client-app-2.7.3-sources.jar; 赠送Maven依赖信息文件:...

    hadoop-yarn-server-resourcemanager-2.6.0-API文档-中文版.zip

    赠送jar包:hadoop-yarn-server-resourcemanager-2.6.0.jar; 赠送原API文档:hadoop-yarn-server-resourcemanager-2.6.0-javadoc.jar; 赠送源代码:hadoop-yarn-server-resourcemanager-2.6.0-sources.jar; 赠送...

    hadoop-mapreduce-client-jobclient-2.6.5-API文档-中英对照版.zip

    赠送jar包:hadoop-mapreduce-client-jobclient-2.6.5.jar; 赠送原API文档:hadoop-mapreduce-client-jobclient-2.6.5-javadoc.jar; 赠送源代码:hadoop-mapreduce-client-jobclient-2.6.5-sources.jar; 赠送...

    hadoop-mapreduce-client-app-2.6.5-API文档-中文版.zip

    赠送jar包:hadoop-mapreduce-client-app-2.6.5.jar; 赠送原API文档:hadoop-mapreduce-client-app-2.6.5-javadoc.jar; 赠送源代码:hadoop-mapreduce-client-app-2.6.5-sources.jar; 赠送Maven依赖信息文件:...

    flink-hadoop-compatibility_2.11-1.10.0-API文档-中文版.zip

    赠送jar包:flink-hadoop-compatibility_2.11-1.10.0.jar; 赠送原API文档:flink-hadoop-compatibility_2.11-1.10.0-javadoc.jar; 赠送源代码:flink-hadoop-compatibility_2.11-1.10.0-sources.jar; 赠送Maven...

    hadoop-mapreduce-client-core-2.7.3-API文档-中英对照版.zip

    赠送jar包:hadoop-mapreduce-client-core-2.7.3.jar; 赠送原API文档:hadoop-mapreduce-client-core-2.7.3-javadoc.jar; 赠送源代码:hadoop-mapreduce-client-core-2.7.3-sources.jar; 赠送Maven依赖信息文件:...

    hadoop-mapreduce-client-app-2.6.5-API文档-中英对照版.zip

    赠送jar包:hadoop-mapreduce-client-app-2.6.5.jar; 赠送原API文档:hadoop-mapreduce-client-app-2.6.5-javadoc.jar; 赠送源代码:hadoop-mapreduce-client-app-2.6.5-sources.jar; 赠送Maven依赖信息文件:...

    hadoop-yarn-server-common-2.6.5-API文档-中文版.zip

    赠送jar包:hadoop-yarn-server-common-2.6.5.jar; 赠送原API文档:hadoop-yarn-server-common-2.6.5-javadoc.jar; 赠送源代码:hadoop-yarn-server-common-2.6.5-sources.jar; 赠送Maven依赖信息文件:hadoop-...

    hadoop-common-2.6.0-bin-master.zip

    `hadoop-common-2.6.0-bin-master.zip` 是一个针对Hadoop 2.6.0版本的压缩包,特别适用于在Windows环境下进行本地开发和测试。这个版本的Hadoop包含了对Windows系统的优化,比如提供了`winutils.exe`,这是在Windows...

    hadoop-yarn-api-2.7.3-API文档-中英对照版.zip

    赠送jar包:hadoop-yarn-api-2.7.3.jar; 赠送原API文档:hadoop-yarn-api-2.7.3-javadoc.jar; 赠送源代码:hadoop-yarn-api-2.7.3-sources.jar; 赠送Maven依赖信息文件:hadoop-yarn-api-2.7.3.pom; 包含翻译后...

    hadoop-hdfs-client-2.9.1-API文档-中英对照版.zip

    赠送jar包:hadoop-hdfs-client-2.9.1.jar; 赠送原API文档:hadoop-hdfs-client-2.9.1-javadoc.jar; 赠送源代码:hadoop-hdfs-client-2.9.1-sources.jar; 赠送Maven依赖信息文件:hadoop-hdfs-client-2.9.1.pom;...

    hadoop-yarn-server-web-proxy-2.6.0-API文档-中文版.zip

    赠送jar包:hadoop-yarn-server-web-proxy-2.6.0.jar; 赠送原API文档:hadoop-yarn-server-web-proxy-2.6.0-javadoc.jar; 赠送源代码:hadoop-yarn-server-web-proxy-2.6.0-sources.jar; 赠送Maven依赖信息文件:...

    hadoop-yarn-client-2.6.5-API文档-中英对照版.zip

    赠送jar包:hadoop-yarn-client-2.6.5.jar; 赠送原API文档:hadoop-yarn-client-2.6.5-javadoc.jar; 赠送源代码:hadoop-yarn-client-2.6.5-sources.jar; 赠送Maven依赖信息文件:hadoop-yarn-client-2.6.5.pom;...

    hadoop插件apache-hadoop-3.1.0-winutils-master.zip

    标题中的"apache-hadoop-3.1.0-winutils-master.zip"是一个针对Windows用户的Hadoop工具包,它包含了运行Hadoop所需的特定于Windows的工具和配置。`winutils.exe`是这个工具包的关键组件,它是Hadoop在Windows上的一...

Global site tag (gtag.js) - Google Analytics