hive在YARN下执行mr任务问题

错误和解决方案都可以参加该链接：
http://grokbase.com/p/cloudera/cdh-user/126wqvfwyt/hive-refuses-to-work-with-yarn

现象就是，当hive启动mr任务是报错如下：

引用

Error during job, obtaining debugging information...
Examining task ID: task_1347634167505_0004_m_000000 (and more) from job job_1347634167505_0004
Exception in thread "Thread-19" java.lang.IllegalArgumentException: Does not contain a valid host:port authority: local
        at org.apache.hadoop.net.NetUtils.createSocketAddr(NetUtils.java:206)
        at org.apache.hadoop.net.NetUtils.createSocketAddr(NetUtils.java:158)
        at org.apache.hadoop.net.NetUtils.createSocketAddr(NetUtils.java:147)
        at org.apache.hadoop.hive.ql.exec.JobTrackerURLResolver.getURL(JobTrackerURLResolver.java:42)
        at org.apache.hadoop.hive.ql.exec.JobDebugger.showJobFailDebugInfo(JobDebugger.java:209)
        at org.apache.hadoop.hive.ql.exec.JobDebugger.run(JobDebugger.java:92)
        at java.lang.Thread.run(Thread.java:722)
Execution failed with exit status: 2

链接中的解释很清楚

引用

There are actually two failures here:

1) The MR job that Hive launched on your cluster failed for some reason. I
can't determine why based on the information provided. I recommend trying
to locate the task logs for the failed tasks on the cluster.

2) When a job fails Hive attempts to automatically retrieve the task logs
from the JobTracker's TaskLogServlet. This service doesn't exist in MR2,
which is why Hive is throwing an exception (either because
mapred.job.tracker is undefined, or because it can't find the
TaskLogServlet service running on the machine that mapred.job.tracker
points to). This is a known issue and one that we plan to address in the
next release of CDH.

解决方法也很详细，就是在hive-site.xml中添加设置，

引用

In the meantime I recommend doing the following if you need to run Hive on
MR2:
* Keep Hive happy by setting mapred.job.tracker to a bogus value.
* Disable task log retrieval by setting
hive.exec.show.job.failure.debug.info=false

然后就ok啦。

评论

1 楼 cloudeagle_bupt 2013-03-04  

求教楼主: 在hadoop-2.0.2时，该方法可行，但当换成hadoop2.0.3-alpha时，报错如下:
Exception in thread "Thread-30" java.lang.IllegalArgumentException: Does not contain a valid host:port authority: ignorethis
        at org.apache.hadoop.net.NetUtils.createSocketAddr(NetUtils.java:208)
        at org.apache.hadoop.net.NetUtils.createSocketAddr(NetUtils.java:160)
        at org.apache.hadoop.net.NetUtils.createSocketAddr(NetUtils.java:149)
        at org.apache.hadoop.hive.ql.exec.JobTrackerURLResolver.getURL(JobTrackerURLResolver.java:42)
        at org.apache.hadoop.hive.ql.exec.JobDebugger.showJobFailDebugInfo(JobDebugger.java:209)
        at org.apache.hadoop.hive.ql.exec.JobDebugger.run(JobDebugger.java:92)
        at java.lang.Thread.run(Thread.java:722)

请问有啥办法没有啊?