在虚拟机中搭建了一个伪分布式的hadoop环境,基于ubuntu13.04,可以在命令行中通过hadoop jar运行自带wordcount。另外,主机运行的是win7+eclipse4.3,已将hadoop的源码导入eclipse,并且eclipse的hadoop插件也已经编译安装好了,但是在主机的eclipse中run on hadoop运行wordcount,却总是报一个异常
java.io.IOException: Failed to set permissions of path: \XX..XX\.staging to 0700 at org.apache.hadoop.fs.FileUtil.checkReturnValue(FileUtil.java:691) at org.apache.hadoop.fs.FileUtil.setPermission(FileUtil.java:664) at org.apache.hadoop.fs.RawLocalFileSystem.setPermission(RawLocalFileSystem.java:514) at org.apache.hadoop.fs.RawLocalFileSystem.mkdirs(RawLocalFileSystem.java:349) at org.apache.hadoop.fs.FilterFileSystem.mkdirs(FilterFileSystem.java:193) at org.apache.hadoop.mapreduce.JobSubmissionFiles.getStagingDir(JobSubmissionFiles.java:126) at org.apache.hadoop.mapred.JobClient$2.run(JobClient.java:942) at org.apache.hadoop.mapred.JobClient$2.run(JobClient.java:1) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:396) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1190) at org.apache.hadoop.mapred.JobClient.submitJobInternal(JobClient.java:936) at org.apache.hadoop.mapreduce.Job.submit(Job.java:550) at org.apache.hadoop.mapreduce.Job.waitForCompletion(Job.java:580) at com.shidan.mapred.CombinerTest.main(CombinerTest.java:41)
并且运行的时候会在主机的本地磁盘上创建上述path的目录结构,该path应该是在eclipse所在的磁盘分区根目录下的,比如说我的eclipse是安装在D盘里面的,所以上面的路径在我电脑上就是D:\XX..XX\.staging
(该路径其实是hadoop中配置项mapreduce.jobtracker.staging.root.dir的值)
很是纳闷,明明是run on hadoop的呀,跟我的本地文件系统有什么关系,不解。google一番后,有人提到run on hadoop其实会在本地运行job,只不过会用到远程的hdfs,这样的话,倒是可以理解为什么会报这个错误,因为尝试在windows文件系统中去设置linux中的权限。既然在本地跑Job,那跟我直接在eclipse里面run as java application有什么区别呢?于是直接run as application,发现也报同样的错误,好像没有什么区别!仔细看该异常,是在FileUtil类中checkReturnValue方法里面抛出来的,于是乎,直接将该方法里面的代码注释掉,再run on hadoop,发现可以正常输出结果啦,再试试run as application,也可以的。
虽然job能跑起来了,但是run on hadoop比run as java application有什么不同,还是没有搞清楚,暂且记下吧!
================分割线===========================
run on hadoop 和 run as java application的区别在我的另一篇文章中已经解释啦~
===>传送门
相关推荐
eclipse运行作业 Failed to set permissions of path: \tmp\hadoop-admin\mapred\staging\Administrator-1506477061\.staging to 0700 :Windows环境下的Hadoop TaskTracker无法正常启动 包括0.20.204、0.20.205、...
eclipse远程调试hadoop时 报出eclipse Hadoop Failed to set permissions of path错误 修改hadoop core包中FileUtil java文件 里面有checkReturnValue方法 将代码throw new IOException "Failed to set ...
Failed to set permissions of path: \tmp\hadoop-Administrator,的解决方法,更换hadoop-core-1.0.2-modified.jar包
解决nutch在window系统安装报错问题Exception in thread "main" java.io.IOException: Failed to set permissions of path: \tmp\hadoop-tom\mapred\staging\tom1698089073\.staging to 0700,替换jar包OK!
今天在同步数据的时候提示rsync: failed to set times on “xxxx”: Operation not permitted,一般来说要不是服务器时间不对或者权限没有设置好,下面软件开发网小编就为大家整理了一些资料 第一种方法: 出现rsync...
12/04/24 15:32:44 ERROR security.UserGroupInformation: PriviledgedActionException as:Administrator cause:java.io.IOException: Failed to set permissions of path: \tmp\hadoop-Administrator\mapred\...
windows下搭建nutch会...java.io.IOException: Failed to set permissions of path: \tmp\hadoop-Administrator\mapred\staging\Administrator606301699\.staging to 0700 此jar包是已经注释好 的然后从新编译的jar。
Exception in thread "main" java.io.IOException: Failed to set permissions of path: \tmp\hadoop-Administrator\mapred\staging\Administrator1702422322\.staging to 0700 这个错误是win中开发特有的错误,文件...
ERROR security.UserGroupInformation: PriviledgedActionException as:chuck cause:java.io.IOException: Failed to set permissions of path: 分析问题: 属于windows下的eclipse中访问的权限问题,需要重新打包...
Win环境下本地运行hadoop1版本出现访问权限问题,会遇到如下报错: Exception in thread "main" java.io.IOException: Failed to set permissions of path: \tmp\hadoop-Administrator\mapred\staging\...
解决nutch 1.9在cgywin中运行报Injector: java.io.IOException: Failed to set permissions of path: \tmp\hadoop-Administrator\mapred\staging\Administrator139727905\.staging to 0700错误的问题,即修改hadoop-...
重新编译的hadoop-1.2.0的核心jar包,解决****Exception in thread "main" java.io.IOException: Failed to set permissions of path:****windows下的文件权限问题。
Exception in thread "main" java.io.IOException: Failed to set permissions of path: \tmp\hadoop-ywf\mapred\staging\ywf1760469788\.staging to 0700 win7下报错,需要注释掉checkReturnValue的方法体
在windows下,使用hadoop-1.0.4自带的hadoop-core-1.0.4.jar时,会抛出权限异常:cause:java.io.IOException: Failed to set permissions of path: \tmp\hadoop-Administrator\mapred\staging\Administrator-...
Failed to save ‘index.jsx’: Insufficient permissions. Select ‘Retry as Sudo’ to retry as superuser. EACCES: permission denied 解决方法:授予权限 sudo chmod -R 777 . ## 为当前目录及子目录搜于读写...