一个hadoop问题 - ITeye问答

0 0

一个hadoop问题10

刚刚学习hadoop，搭建起集群在运行hadoop自带例子时报如下错误。请高手帮我解决下
hadoop jar c:/hadoop/hadoop-0.20.2/hadoop-0.20.2-examples.jar wordcount in/test.txt out 12/06/08 15:57:08 INFO input.FileInputFormat: Total input paths to process : 1 12/06/08 15:57:08 INFO mapred.JobClient: Running job: job_201206081518_0001 12/06/08 15:57:09 INFO mapred.JobClient: map 0% reduce 0% 12/06/08 15:57:18 INFO mapred.JobClient: map 100% reduce 0% 12/06/08 15:57:32 INFO mapred.JobClient: Task Id : attempt_201206081518_0001_r_000000_0, Status : FAILED java.io.IOException: Can not get the relative path: base = hdfs://MASTER:9000/user/Administrator/out/_temporary/_attempt_201206081518_0001_r_000000_0 child = hdfs://master:9000/user/Administrator/out/_temporary/_attempt_201206081518_0001_r_000000_0/part-r-00000 at org.apache.hadoop.mapreduce.lib.output.FileOutputCommitter.getFinalPath(FileOutputCommitter.java:200) at org.apache.hadoop.mapreduce.lib.output.FileOutputCommitter.moveTaskOutputs(FileOutputCommitter.java:146) at org.apache.hadoop.mapreduce.lib.output.FileOutputCommitter.moveTaskOutputs(FileOutputCommitter.java:165) at org.apache.hadoop.mapreduce.lib.output.FileOutputCommitter.commitTask(FileOutputCommitter.java:118) at org.apache.hadoop.mapred.Task.commit(Task.java:779) at org.apache.hadoop.mapred.Task.done(Task.java:691) at org.apache.hadoop.mapred.ReduceTask.run(ReduceTask.java:414) at org.apache.hadoop.mapred.Child.main(Child.java:170) 12/06/08 15:57:47 INFO mapred.JobClient: Task Id : attempt_201206081518_0001_r_000000_1, Status : FAILED java.io.IOException: Can not get the relative path: base = hdfs://MASTER:9000/user/Administrator/out/_temporary/_attempt_201206081518_0001_r_000000_1 child = hdfs://master:9000/user/Administrator/out/_temporary/_attempt_201206081518_0001_r_000000_1/part-r-00000 at org.apache.hadoop.mapreduce.lib.output.FileOutputCommitter.getFinalPath(FileOutputCommitter.java:200) at org.apache.hadoop.mapreduce.lib.output.FileOutputCommitter.moveTaskOutputs(FileOutputCommitter.java:146) at org.apache.hadoop.mapreduce.lib.output.FileOutputCommitter.moveTaskOutputs(FileOutputCommitter.java:165) at org.apache.hadoop.mapreduce.lib.output.FileOutputCommitter.commitTask(FileOutputCommitter.java:118) at org.apache.hadoop.mapred.Task.commit(Task.java:779) at org.apache.hadoop.mapred.Task.done(Task.java:691) at org.apache.hadoop.mapred.ReduceTask.run(ReduceTask.java:414) at org.apache.hadoop.mapred.Child.main(Child.java:170) 12/06/08 15:58:02 INFO mapred.JobClient: Task Id : attempt_201206081518_0001_r_000000_2, Status : FAILED java.io.IOException: Can not get the relative path: base = hdfs://MASTER:9000/user/Administrator/out/_temporary/_attempt_201206081518_0001_r_000000_2 child = hdfs://master:9000/user/Administrator/out/_temporary/_attempt_201206081518_0001_r_000000_2/part-r-00000 at org.apache.hadoop.mapreduce.lib.output.FileOutputCommitter.getFinalPath(FileOutputCommitter.java:200) at org.apache.hadoop.mapreduce.lib.output.FileOutputCommitter.moveTaskOutputs(FileOutputCommitter.java:146) at org.apache.hadoop.mapreduce.lib.output.FileOutputCommitter.moveTaskOutputs(FileOutputCommitter.java:165) at org.apache.hadoop.mapreduce.lib.output.FileOutputCommitter.commitTask(FileOutputCommitter.java:118) at org.apache.hadoop.mapred.Task.commit(Task.java:779) at org.apache.hadoop.mapred.Task.done(Task.java:691) at org.apache.hadoop.mapred.ReduceTask.run(ReduceTask.java:414) at org.apache.hadoop.mapred.Child.main(Child.java:170) 12/06/08 15:58:20 INFO mapred.JobClient: Job complete: job_201206081518_0001 12/06/08 15:58:20 INFO mapred.JobClient: Counters: 12 12/06/08 15:58:20 INFO mapred.JobClient: Job Counters 12/06/08 15:58:20 INFO mapred.JobClient: Launched reduce tasks=4 12/06/08 15:58:20 INFO mapred.JobClient: Launched map tasks=1 12/06/08 15:58:20 INFO mapred.JobClient: Data-local map tasks=1 12/06/08 15:58:20 INFO mapred.JobClient: Failed reduce tasks=1 12/06/08 15:58:20 INFO mapred.JobClient: FileSystemCounters 12/06/08 15:58:21 INFO mapred.JobClient: HDFS_BYTES_READ=25 12/06/08 15:58:21 INFO mapred.JobClient: FILE_BYTES_WRITTEN=93 12/06/08 15:58:21 INFO mapred.JobClient: Map-Reduce Framework 12/06/08 15:58:21 INFO mapred.JobClient: Combine output records=5 12/06/08 15:58:21 INFO mapred.JobClient: Map input records=2 12/06/08 15:58:21 INFO mapred.JobClient: Spilled Records=5 12/06/08 15:58:21 INFO mapred.JobClient: Map output bytes=45 12/06/08 15:58:21 INFO mapred.JobClient: Combine input records=5 12/06/08 15:58:21 INFO mapred.JobClient: Map output records=5
问题补充
我觉得是hdfs://MASTER:9000/user/Administrator/out/_temporary/_attempt_201206081518_0001_r_000000_0 child = hdfs://master:9000/user/Administrator/out/_temporary/_attempt_201206081518_0001_r_000000_0/part-r-00000 at
master 大小写的问题。但是不知道在哪里改?

Hadoop

2012年6月14日 11:44

yaoxiaowei0909
12
0 0 3

2个答案按时间排序按投票排序

0 0

采纳的答案

能不能正常启动？或许是hdfs://MASTER:9000这个所在的xml格式或者其他地方有错误

2012年6月14日 14:20

baikunhu
58
0 0 3

1条评论

0 0

确定你的hadoop正常启动，如果是为分布式的话，用jps命令查看一下五个进程是否正常启动：
2407 Jps
1914 DataNode
1751 NameNode
2075 SecondaryNameNode
2290 TaskTracker
2139 JobTracker

在Hadoop中用于监控集群健康状态的Web界面，可以访问连接确保你的hadoop启动成功
http://localhost:50030/ – web UI for MapReduce job tracker(s)
http://localhost:50070/ – web UI for HDFS name node(s)

2012年6月21日 13:45

zhaojinll
30
0 0 0