0 0

hadoop 验证map/reduce出错 ERROR security.UserGroupInformation: PriviledgedActionExcept5

在 hadoop2集群  master1上, 执行 HADOOP自带的例子, wordcount 包,命令如下
cd /usr/local/hadoop/hadoop-2.0.2-alpha/share/hadoop/mapreduce
hadoop jar hadoop-mapreduce-examples-2.0.2-alpha.jar wordcount hdfs://192.168.1.33:9000/input hdfs://192.168.1.33:9000/output
出现错误:
13/06/24 05:48:06 ERROR security.UserGroupInformation: PriviledgedActionException as:hadoop (auth:SIMPLE) cause:java.io.IOException: Cannot initialize Cluster. Please check your configuration for mapreduce.framework.name and the correspond server addresses.
java.io.IOException: Cannot initialize Cluster. Please check your configuration for mapreduce.framework.name and the correspond server addresses.
        at org.apache.hadoop.mapreduce.Cluster.initialize(Cluster.java:121)
        at org.apache.hadoop.mapreduce.Cluster.<init>(Cluster.java:83)
        at org.apache.hadoop.mapreduce.Cluster.<init>(Cluster.java:76)
        at org.apache.hadoop.mapreduce.Job$10.run(Job.java:1188)
        at org.apache.hadoop.mapreduce.Job$10.run(Job.java:1184)
        at java.security.AccessController.doPrivileged(Native Method)
        at javax.security.auth.Subject.doAs(Subject.java:415)
        at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1367)
        at org.apache.hadoop.mapreduce.Job.connect(Job.java:1183)
        at org.apache.hadoop.mapreduce.Job.submit(Job.java:1212)
        at org.apache.hadoop.mapreduce.Job.waitForCompletion(Job.java:1236)
        at org.apache.hadoop.examples.WordCount.main(WordCount.java:84)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:601)
        at org.apache.hadoop.util.ProgramDriver$ProgramDescription.invoke(ProgramDriver.java:72)
        at org.apache.hadoop.util.ProgramDriver.driver(ProgramDriver.java:144)
        at org.apache.hadoop.examples.ExampleDriver.main(ExampleDriver.java:68)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:601)
        at org.apache.hadoop.util.RunJar.main(RunJar.java:208)
   但是我通过start-all.sh启动之后,jps查看 master机器上:ResourceManager和NameNode都已启动,slave机器上NodeManager和DataNode也已经启动了,查看启动日志均无报错,但是验证map/reduce时就报这个错误,请指点迷津,不胜感谢
2013年6月25日 10:35

3个答案 按时间排序 按投票排序

0 0

修改hdfs-site.xml(如果原来没有就建一个),如下:
<property> 
  <name>dfs.permissions</name> 
  <value>false</value> 
</property> 
可以参考http://sgq0085.iteye.com/admin/blogs/1851294

2013年7月05日 15:42
0 0

把IP改成机器名试试

2013年7月04日 12:30
0 0

权限问题,Hadoop有个配置,把权限配置关掉再试试。

2013年6月25日 11:23

相关推荐

    hadoop-core-1.1.1.jar

    12/04/24 15:32:44 ERROR security.UserGroupInformation: PriviledgedActionException as:Administrator cause:java.io.IOException: Failed to set permissions of path: \tmp\hadoop-Administrator\mapred\...

    Windows平台下Hadoop的Map/Reduce开发

    在Windows平台上进行Hadoop的Map/Reduce开发可能会比在Linux环境下多一些挑战,但通过详细的步骤和理解Map/Reduce的工作机制,开发者可以有效地克服这些困难。以下是对标题和描述中涉及知识点的详细说明: **Hadoop...

    hadoop-native-lib-x64-2.7.3.tar.gz

    hadoop 2.7.3 的本地接口库,snappy 使用的是1.1.3版本。使用时直接把包解压后放到$HADOOP_HOME/lib/native目录下即可。 [root@pseudo ~]# hadoop checknative -a 17/09/11 14:30:49 WARN bzip2.Bzip2Factory: ...

    hadoop中map/reduce

    MapReduce的设计理念源于Google的同名论文,它通过将大规模数据处理任务分解为两个阶段:Map(映射)和Reduce(化简),使得海量数据能够在多台计算机上并行处理,极大地提高了数据处理效率。 Map阶段是数据处理的...

    HDFS上传文件报错org.apache.hadoop.fs.ChecksumException: Checksum error: file:/hyk/data/hyk.txt

    当从本地上传文件到HDFS中...org.apache.hadoop.fs.ChecksumException: Checksum error: file:/hyk/data/hyk.txt [root@node01 data]# hadoop fs -put hyk.txt /hyk/test 20/02/18 12:54:39 INFO fs.FSInputChecker: Fo

    hadoop常见问题及解决办法

    问题描述:在reduce阶段,Hadoop抛出Shuffle Error:Exceeded MAX_FAILED_UNIQUE_FETCHES错误。 解决办法:该问题是由于reduce预处理阶段shuffle时获取已完成的map输出失败次数超过上限造成的,上限默认为5。解决...

    hadoop1.0 Failed to set permissions of path 解决方案

    ERROR org.apache.hadoop.mapred.TaskTracker: Can not start task tracker because java.io.IOException: Failed to set permissions of path: \tmp\hadoop-admin \mapred\local\ttprivate to 0700 at org.apache...

    hadoop-2.6.0-hadoop.dll-winutils.exe

     at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1628)  at org.apache.hadoop.mapreduce.Job.submit(Job.java:1293)  at org.apache.hadoop.mapreduce.Job....

    hadoop 2.7.6 eclipse插件

    18/05/25 19:51:49 INFO mapreduce.Job: map 0% reduce 0% 18/05/25 19:52:20 INFO mapreduce.Job: map 100% reduce 0% 18/05/25 19:52:29 INFO mapreduce.Job: map 100% reduce 100% 18/05/25 19:52:31 INFO ...

    hadoop-3.0.0 win工具包 hadoop.dll winutils.exe

    util.Shell (Shell.java:(694)) - Did not find ...java.io.FileNotFoundException: Could not locate Hadoop executable: E:\hadoop-3.0.2\bin\winutils.exe -see https://wiki.apache.org/hadoop/WindowsProblems

    远程调用执行Hadoop Map/Reduce

    例如,`org.apache.hadoop.mapred.MapTask`和`org.apache.hadoop.mapreduce.ReduceTask`分别对应Map和Reduce任务的实现,开发者可以通过阅读这些源码了解任务执行的详细流程。 7. **工具集成**:有许多开源工具可以...

    hadoop-core-0.20.2-cdh3u3.jar

    export CLASSPATH="$JAVA_HOME/lib/dt.jar:$JAVA_HOME/lib/tools.jar:${HADOOP_HOME}/lib/commons-logging-1.0.4.jar:${HADOOP_HOME}/lib/guava-r09-jarjar.jar:${HADOOP_HOME}/hadoop-core-0.20.2-cdh3u3.jar:/usr/...

    hadoop-core-1.2.1.jar

    解决windows下远程调试hadoop权限问题( ERROR security.UserGroupInformation: PriviledgedActionException as:Administrator cause:java.io.IOException: Failed to set permissions of path:)重新编译了hadoop-...

    【Spark】六、org.apache.hadoop.security.AccessControlException

    Exception in thread main org.apache.hadoop.security.AccessControlException: Permission denied: user=L.MOON, access=WRITE, inode=/user/lsy/result1/_temporary/0:lsy:supergroup:drwxr-xr-x Caused by: org....

    hadoop配置文件参数详解1

    * hadoop.security.group.mapping:组内用户的列表的类设定,default value 是 org.apache.hadoop.security.ShellBasedUnixGroupsMapping。 * hadoop.security.authorization:authorization,default value 是 ...

    hadoop安装与配置 Hadoop的安装与配置可以分成几个主要步骤: 1. 安装Java 2. 下载Hadoop 3. 配

    hadoop安装与配置 hadoop安装与配置 Hadoop的安装与配置可以分成几个主要步骤: 1. 安装Java 2. 下载Hadoop 3. 配置Hadoop 4. 格式化Hadoop文件系统 5. 启动Hadoop 以下是基于Linux系统的简化安装与配置步骤: 1. ...

    winutils.exe:解决hadoop在windows运行出现的bug

    如果出现如下bug:“Could not locate executable null\bin\winutils.exe in the Hadoop binaries”,则下载该文件,放入hadoop的bin文件夹下,并设置环境变量HADOOP_HOME:F:\hadoop2.7.x即可。

Global site tag (gtag.js) - Google Analytics