分析”failed to report status for 602 seconds”

测试下载一个大文件是否会出现Task attempt_201010291725_0041_m_000000_1 failed to report status for 602 seconds. Killing!
1、找一个比较大的文件：http://down1.520sys.cn/down/tomato/FQ_GhostXPsp3_1.5_%B4%BF%BE%BB%B0%E6.iso ，这个文件大约 600M.
2、做一个简单的MapReduce程序，实现下载-上述链接。
   a: 创建一个文件，此文件就存放所要下载的URL.并且上传到HDFS.
   b: 创建一个DownloadMapper类，该类extends Mapper，在这个类中实现下载URL,把所下载的内容直接写入到HDFS中。
   c：在主函数DownloadMain中启动downloadJob.
3、启动download.job
root@se-test1-desktop:~/hadoop/program# tail -f dowload.log
10/11/09 12:58:55 INFO main.DownloadMain: downloadMain start .....
10/11/09 12:58:55 INFO main.DownloadMain: /tmp/ddow copy successfully
10/11/09 12:58:55 WARN mapred.JobClient: Use GenericOptionsParser for parsing the arguments. Applications should implement Tool for the same.
10/11/09 12:59:04 INFO input.FileInputFormat: Total input paths to process : 1
10/11/09 12:59:10 INFO mapred.JobClient: Running job: job_201010291725_0041
10/11/09 12:59:11 INFO mapred.JobClient:  map 0% reduce 0%
10/11/09 12:59:29 INFO mapred.JobClient:  map 100% reduce 0%
10/11/09 13:09:33 INFO mapred.JobClient:  map 0% reduce 0%
10/11/09 13:09:35 INFO mapred.JobClient: Task Id : attempt_201010291725_0041_m_000000_0, Status : FAILED
Task attempt_201010291725_0041_m_000000_0 failed to report status for 602 seconds. Killing!
attempt_201010291725_0041_m_000000_0: Total time for which application threads were stopped: 0.0020320 seconds
attempt_201010291725_0041_m_000000_0: Total time for which application threads were stopped: 0.0017580 seconds
attempt_201010291725_0041_m_000000_0: Total time for which application threads were stopped: 0.0003120 seconds
attempt_201010291725_0041_m_000000_0: Total time for which application threads were stopped: 0.0001860 seconds
attempt_201010291725_0041_m_000000_0: Total time for which application threads were stopped: 0.0004730 seconds
attempt_201010291725_0041_m_000000_0: Total time for which application threads were stopped: 0.0015730 seconds
attempt_201010291725_0041_m_000000_0: Total time for which application threads were stopped: 0.0002630 seconds
attempt_201010291725_0041_m_000000_0: Total time for which application threads were stopped: 0.0004870 seconds
attempt_201010291725_0041_m_000000_0: Total time for which application threads were stopped: 0.0005290 seconds
attempt_201010291725_0041_m_000000_0: Total time for which application threads were stopped: 0.0008870 seconds
attempt_201010291725_0041_m_000000_0: Total time for which application threads were stopped: 0.0005540 seconds
attempt_201010291725_0041_m_000000_0: Total time for which application threads were stopped: 0.0005600 seconds
attempt_201010291725_0041_m_000000_0: Total time for which application threads were stopped: 0.0009300 seconds

4、从日志中可以看出，当下载一个比较大的文件时，需要一定的时间，当这个时间超过配置文件所配置的（mapred.task.timeout--默认600）时间时，就会出现”failed to report status for 602 seconds. Killing!“。
总结 ：经过上述的一个测试我们看出：
      a:在我们自己的MapReduce程序当中所做的工作尽量不要超过默认配置的时间。

评论

1 楼 asd51731 2013-09-18  

请问一下，您说“在我们自己的MapReduce程序当中所做的工作尽量不要超过默认配置的时间”如何进行保证，能不能给一个大致的指导？谢谢