`
bupt04406
  • 浏览: 350169 次
  • 性别: Icon_minigender_1
  • 来自: 杭州
社区版块
存档分类
最新评论

Too many fetch failures

阅读更多

 

http://lucene.472066.n3.nabble.com/Reg-Too-many-fetch-failures-Error-td4037975.html

 

http://www.slideshare.net/cloudera/hadoop-troubleshooting-101-kate-ting-cloudera

 

 

 

 http://lucene.472066.n3.nabble.com/Reg-Too-many-fetch-failures-Error-td4037975.html

 

As you may be aware this means the reduces are unable to fetch intermediate data from TaskTrackers that ran map tasks – you can try:

* increasing tasktracker.http.threads so there are more threads to handle fetch requests from reduces.

* decreasing mapreduce.reduce.parallel.copies : so fewer copy / fetches are performed in parallel

 

It could also be due to a temporary DNS issue.

 

 

cdh3u4:

TaskTracker里面: 

workerThreads = conf.getInt("tasktracker.http.threads", 40);

 

ReduceTask里面: 

this.numCopiers = conf.getInt("mapred.reduce.parallel.copies", 5);

 

 

 http://lucene.472066.n3.nabble.com/Error-Too-Many-Fetch-Failures-td3990324.html

 

 $ cat /proc/sys/net/core/somaxconn 

1024

$ ulimit -n 

131072

 

 

 

 

 

  • 大小: 63 KB
分享到:
评论

相关推荐

    Hadoop使用常见问题以及解决方法

    Too many fetch-failures 是另一个常见的问题,主要是由于结点间的连通不够全面。解决方法是检查 `/etc/hosts` 文件,确保本机 IP 对应服务器名,并包含所有服务器的 IP 和服务器名。同时,检查 `.ssh/authorized_...

    hadoop出错解决方法

    二、Too many fetch-failures 这个错误主要是由于结点间的连通不够全面。解决方法是:1) 检查 /etc/hosts 文件,确保本机 IP 对应服务器名,包含所有服务器的 IP 和服务器名。2) 检查 .ssh/authorized_keys 文件,...

    Hadopp集群运用过程中碰到的问题

    - **错误现象3:Too many fetch-failures** - 原因:节点间网络连接不稳定或配置不完整。 - 解决方法:检查并更新/etc/hosts文件,确保所有服务器的IP和主机名对应无误,同时确认.ssh/authorized_keys文件包含了...

    Hadoop入门实战手册

    - **Too many fetch-failures**:同样与网络问题有关,可以调整MapReduce的配置来减少这类错误的发生。 - **能够启动datanode,但无法访问,也无法结束的错误**:可能是由于防火墙或安全策略阻止了访问。 - **java....

    微软内部资料-SQL性能优化3

    After a transaction commits, its effects will persist even if there are system failures. Consistency and isolation are the most important in describing SQL Server’s locking model. It is up to the ...

Global site tag (gtag.js) - Google Analytics