Hadoop 异常处理

错误处理：

l  org.apache.hadoop.ipc.RemoteException: org.apache.hadoop.hdfs.server.namenode.SafeModeException: Cannot delete /tmp/hadoop-Administrator/mapred/system. Name node is in safe mode.

手动删除：E:\TMP\hadoop-Administrator\mapred\system

  

Safe mode is ON. The ratio of reported blocks 1.0000 has reached the threshold 0.9990. Safe mode will be turned off automatically in 10 seconds.

正常状态，耐心等待退出Safe mode

10/04/09 16:16:51 INFO ipc.Client: Retrying connect to server: /192.168.2.79:9000. Already tried 0 time(s).

Hadoop 没有启动

SSH问题：

http://snowyowls.blogbus.com/logs/2575305.html

 

新建账户bmb

创建passphraseless ssh失败

ssh localhost

查看日志：tail  -f /var/log/secure

Dec 14 14:12:53 localhost sshd[19845]: Authentication refused: bad ownership or modes for file /home/bmb/.ssh/authorized_keys

 

原因：authorized_keys需要是644权限

chmod 644 authorized_keys

同时，.ssh的上级目录需要700权限

 

Hadoop + Hbase Map reduce,   Map Task 异常（无解）

2010-04-13 21:17:40,363 WARN org.mortbay.log: Ignore, probably already closed
java.io.IOException: The client is stopped
at org.apache.hadoop.hbase.ipc.HBaseClient.getConnection(HBaseClient.java:809)
at org.apache.hadoop.hbase.ipc.HBaseClient.call(HBaseClient.java:712)
at org.apache.hadoop.hbase.ipc.HBaseRPC$Invoker.invoke(HBaseRPC.java:333)
at $Proxy2.close(Unknown Source)
at org.apache.hadoop.hbase.client.ScannerCallable.close(ScannerCallable.java:101)
at org.apache.hadoop.hbase.client.ScannerCallable.call(ScannerCallable.java:72)
at org.apache.hadoop.hbase.client.ScannerCallable.call(ScannerCallable.java:38)
at org.apache.hadoop.hbase.client.HConnectionManager$TableServers.getRegionServerWithRetries(HConnectionManager.java:1035)
at org.apache.hadoop.hbase.client.HTable$ClientScanner.nextScanner(HTable.java:1899)
at org.apache.hadoop.hbase.client.HTable$ClientScanner.next(HTable.java:2017)
at org.apache.hadoop.hbase.mapreduce.TableInputFormatBase$TableRecordReader.nextKeyValue(TableInputFormatBase.java:207)
at org.apache.hadoop.mapred.MapTask$NewTrackingRecordReader.nextKeyValue(MapTask.java:423)
at org.apache.hadoop.mapreduce.MapContext.nextKeyValue(MapContext.java:67)
at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:143)
at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:621)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:305)
at org.apache.hadoop.mapred.Child.main(Child.java:170)

 

-----------------------------------------------------------------------------

当集群建立后，想推倒重来

bin/hadoop namenode -format

重启后

bin/start-all.sh

 

日志报错（原因是DataNode启动不正常，因为DataNode所在的目录没有清空，/tmp/hadoop-iic/目录是HDFS）

2.168.2.79:43702: error: java.io.IOException: File /tmp/hadoop-iic/mapred/system/jobtracker.info could only be replicated to 0 nodes, instead of 1

 

查看DataNode的错误日志：

java.io.IOException: Incompatible namespaceIDs in /home/iic/hadoop-0.20.2/hdfs-data: namenode namespaceID = 1444851414; datanode namespaceID = 620054427

 

清空集群里，所有目录/home/iic/hadoop-0.20.2/hdfs-data下的东西

 

 

the hadoop-site.xml cannot resolve host names from a remote node.

http://www.manning-sandbox.com/thread.jspa?messageID=90693

 

网络配置

vi /etc/hosts

vi /etc/sysconfig/network

 

Redhat关闭SendMail

chkconfig sendmail off

could only be replicated to 0 nodes, instead of 1

http://wiki.apache.org/hadoop/HowToSetupYourDevelopmentEnvironment

等http://your_host:50070/dfshealth.jsp 能够显示，再操作hadoop

Cannot delete /tmp/hadoop-bmb/mapred/system. Name node is in safe mode.

 

 等到Hadoop自动离开Safenode就可以了

Incompatible namespaceIDs in /home/bmb/hadoop-0.20.1/hdfs-data: namenode namespaceID = 835083767; datanode namespaceID = 1431413652

修改DataNode跟NameNode一样：hdfs-data/current/VERSION

java.lang.IllegalArgumentException: Duplicate metricsName:getProtocolVersion

 

 

 

 

评论

1 楼 亦梦亦真 2011-04-12  

你说的那个无解的问题我也遇到过，正在求解中……