`
shidan66
  • 浏览: 65320 次
  • 性别: Icon_minigender_1
  • 来自: 北京
社区版块
存档分类
最新评论

Windows+eclipse中运行job出错:Failed to set permissions of path

阅读更多

        在虚拟机中搭建了一个伪分布式的hadoop环境,基于ubuntu13.04,可以在命令行中通过hadoop jar运行自带wordcount。另外,主机运行的是win7+eclipse4.3,已将hadoop的源码导入eclipse,并且eclipse的hadoop插件也已经编译安装好了,但是在主机的eclipse中run on hadoop运行wordcount,却总是报一个异常

 java.io.IOException: Failed to set permissions of path: \XX..XX\.staging to 0700
	at org.apache.hadoop.fs.FileUtil.checkReturnValue(FileUtil.java:691)
	at org.apache.hadoop.fs.FileUtil.setPermission(FileUtil.java:664)
	at org.apache.hadoop.fs.RawLocalFileSystem.setPermission(RawLocalFileSystem.java:514)
	at org.apache.hadoop.fs.RawLocalFileSystem.mkdirs(RawLocalFileSystem.java:349)
	at org.apache.hadoop.fs.FilterFileSystem.mkdirs(FilterFileSystem.java:193)
	at org.apache.hadoop.mapreduce.JobSubmissionFiles.getStagingDir(JobSubmissionFiles.java:126)
	at org.apache.hadoop.mapred.JobClient$2.run(JobClient.java:942)
	at org.apache.hadoop.mapred.JobClient$2.run(JobClient.java:1)
	at java.security.AccessController.doPrivileged(Native Method)
	at javax.security.auth.Subject.doAs(Subject.java:396)
	at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1190)
	at org.apache.hadoop.mapred.JobClient.submitJobInternal(JobClient.java:936)
	at org.apache.hadoop.mapreduce.Job.submit(Job.java:550)
	at org.apache.hadoop.mapreduce.Job.waitForCompletion(Job.java:580)
	at com.shidan.mapred.CombinerTest.main(CombinerTest.java:41)
 
并且运行的时候会在主机的本地磁盘上创建上述path的目录结构,该path应该是在eclipse所在的磁盘分区根目录下的,比如说我的eclipse是安装在D盘里面的,所以上面的路径在我电脑上就是D:\XX..XX\.staging
    (该路径其实是hadoop中配置项mapreduce.jobtracker.staging.root.dir的值)
    很是纳闷,明明是run on hadoop的呀,跟我的本地文件系统有什么关系,不解。google一番后,有人提到run on hadoop其实会在本地运行job,只不过会用到远程的hdfs,这样的话,倒是可以理解为什么会报这个错误,因为尝试在windows文件系统中去设置linux中的权限。既然在本地跑Job,那跟我直接在eclipse里面run as java application有什么区别呢?于是直接run as application,发现也报同样的错误,好像没有什么区别!仔细看该异常,是在FileUtil类中checkReturnValue方法里面抛出来的,于是乎,直接将该方法里面的代码注释掉,再run on hadoop,发现可以正常输出结果啦,再试试run as application,也可以的。
    虽然job能跑起来了,但是run on hadoop比run as java application有什么不同,还是没有搞清楚,暂且记下吧!
 
================分割线===========================
run on hadoop 和 run as java application的区别在我的另一篇文章中已经解释啦~
分享到:
评论

相关推荐

    hadoop1.0 Failed to set permissions of path 解决方案

    eclipse运行作业 Failed to set permissions of path: \tmp\hadoop-admin\mapred\staging\Administrator-1506477061\.staging to 0700 :Windows环境下的Hadoop TaskTracker无法正常启动 包括0.20.204、0.20.205、...

    hadoop-core-1.2.0解决eclipse Hadoop Failed to set permissions of path错误

    eclipse远程调试hadoop时 报出eclipse Hadoop Failed to set permissions of path错误 修改hadoop core包中FileUtil java文件 里面有checkReturnValue方法 将代码throw new IOException "Failed to set ...

    Failed to set permissions of path: \tmp\hadoop-Administrator

    Failed to set permissions of path: \tmp\hadoop-Administrator,的解决方法,更换hadoop-core-1.0.2-modified.jar包

    解决nutch在window系统安装问题

    解决nutch在window系统安装报错问题Exception in thread "main" java.io.IOException: Failed to set permissions of path: \tmp\hadoop-tom\mapred\staging\tom1698089073\.staging to 0700,替换jar包OK!

    rsync同步时出现rsync: failed to set times on “xxxx”: Operation not permitted

    今天在同步数据的时候提示rsync: failed to set times on “xxxx”: Operation not permitted,一般来说要不是服务器时间不对或者权限没有设置好,下面软件开发网小编就为大家整理了一些资料 第一种方法: 出现rsync...

    hadoop-core-1.1.1.jar

    12/04/24 15:32:44 ERROR security.UserGroupInformation: PriviledgedActionException as:Administrator cause:java.io.IOException: Failed to set permissions of path: \tmp\hadoop-Administrator\mapred\...

    hadoop-core-1.2.2-SNAPSHOT.jar

    windows下搭建nutch会...java.io.IOException: Failed to set permissions of path: \tmp\hadoop-Administrator\mapred\staging\Administrator606301699\.staging to 0700 此jar包是已经注释好 的然后从新编译的jar。

    hadoop-core-1.0.3.jar

    Exception in thread "main" java.io.IOException: Failed to set permissions of path: \tmp\hadoop-Administrator\mapred\staging\Administrator1702422322\.staging to 0700 这个错误是win中开发特有的错误,文件...

    hadoop-core-1.2.1.jar

    ERROR security.UserGroupInformation: PriviledgedActionException as:chuck cause:java.io.IOException: Failed to set permissions of path: 分析问题: 属于windows下的eclipse中访问的权限问题,需要重新打包...

    hadoop访问权限问题 hadoop-core-1.1.2.jar

    Win环境下本地运行hadoop1版本出现访问权限问题,会遇到如下报错: Exception in thread "main" java.io.IOException: Failed to set permissions of path: \tmp\hadoop-Administrator\mapred\staging\...

    hadoop-core-1.2.0.jar

    解决nutch 1.9在cgywin中运行报Injector: java.io.IOException: Failed to set permissions of path: \tmp\hadoop-Administrator\mapred\staging\Administrator139727905\.staging to 0700错误的问题,即修改hadoop-...

    hadoop-core-1.2.0

    重新编译的hadoop-1.2.0的核心jar包,解决****Exception in thread "main" java.io.IOException: Failed to set permissions of path:****windows下的文件权限问题。

    win7下注释掉checkReturnValue方法体后的jar文件

    Exception in thread "main" java.io.IOException: Failed to set permissions of path: \tmp\hadoop-ywf\mapred\staging\ywf1760469788\.staging to 0700 win7下报错,需要注释掉checkReturnValue的方法体

    hadoop-core-1.0.4.jar

    在windows下,使用hadoop-1.0.4自带的hadoop-core-1.0.4.jar时,会抛出权限异常:cause:java.io.IOException: Failed to set permissions of path: \tmp\hadoop-Administrator\mapred\staging\Administrator-...

    mac vacode 出现文件拒绝修改的问题

    Failed to save ‘index.jsx’: Insufficient permissions. Select ‘Retry as Sudo’ to retry as superuser. EACCES: permission denied 解决方法:授予权限 sudo chmod -R 777 . ## 为当前目录及子目录搜于读写...

Global site tag (gtag.js) - Google Analytics