java.io.IOException: Task process exit with nonzero status of 1.
at org.apache.hadoop.mapred.TaskRunner.run(TaskRunner.java:418)
11/03/15 12:54:09 WARN mapred.JobClient: Error reading task outputhttp:.....
该问题主要由hadoop的logs目录无法继续写入新文件造成,清理master和slaver上的logs文件夹后再次下发任务,正常运行。
这是老外的解释:
Apparently, it's an OS limit on the number of sub-directories that can be reated in another directory. In this case, we had 31998 sub-directories uder hadoop/userlogs/, so any new tasks would fail in Job Setup.
From the unix command line, mkdir fails as well:
$ mkdir hadoop/userlogs/testdir
mkdir: cannot create directory `hadoop/userlogs/testdir': Too many links
Difficult to track down because the Hadoop error message gives no hint whasoever. And normally, you'd look in the userlog itself for more info, butin this case the userlog couldn't be created.
分享到:
相关推荐
"Data Analytics with Hadoop: An Introduction for Data Scientists" ISBN: 1491913703 | 2016 | PDF | 288 pages | 7 MB Ready to use statistical and machine-learning techniques across large data sets? ...
- **书名**:《Hadoop:The Definitive Guide》(第二版) - **作者**:Tom White - **前言作者**:Doug Cutting - **出版社**:O'Reilly Media, Inc. - **出版日期**:2010年10月 - **版权**:版权所有 © 2011 Tom...
实战Hadoop:开启通向云计算的捷径
Hadoop: The Definitive Guide, 4th Edition Get ready to unlock the power of your data. With the fourth edition of this comprehensive guide, you’ll learn how to build and maintain reliable, scalable,...
《Hadoop:权威指南》是了解和掌握Apache Hadoop生态系统不可或缺的一本著作。这本书由Tom White撰写,全面深入地介绍了Hadoop的各个组件及其工作原理,对于初学者和专业人士来说都是一份宝贵的参考资料。 Hadoop是...
资源名称:云计算Hadoop:快速部署Hadoop集群内容简介: 近来云计算越来越热门了,云计算已经被看作IT业的新趋势。云计算可以粗略地定义为使用自己环境之外的某一服务提供的可伸缩计算资源,并按使用量付费。可以...
大数据处理框架:Hadoop:Hadoop生态系统概览.docx
大数据处理框架:Hadoop:Hadoop数据存储格式.docx
大数据处理框架:Hadoop:Hadoop集群部署与管理.docx
Apache Hadoop:Hadoop集群运维与优化.docx
大数据处理框架:Hadoop:Hadoop数据处理框架Pig.docx
大数据处理框架:Hadoop:大数据与Hadoop简介.docx
大数据处理框架:Hadoop:Hadoop性能优化与故障排查.docx
大数据处理框架:Hadoop:Hadoop实时数据处理框架Flume.docx
大数据处理框架:Hadoop:HadoopYARN架构与资源管理.docx
Apache Hadoop:Hadoop资源管理器YARN详解.docx
Apache Hadoop:Hadoop数据仓库Hive入门与应用.docx
《Hadoop:The Definitive Guide》是O'REILLY出版社出版的一本关于Apache Hadoop的权威指南,目前流行的是第四版。这本书为读者提供了一个全面的Hadoop学习平台,内容涵盖了如何构建和维护一个既可靠又可扩展的...