Hadoop Oozie学习笔记(七) E0903: Configuration does not have Jobtracker Kerberos异常解决

 

  本博客属原创文章,转载请注明出处:http://guoyunsky.iteye.com/blog/1396801

  欢迎加入Hadoop超级群: 180941958

 

     最近开始监控Oozie所有workflow job,比如跟踪所有workflow job,有失败的job就邮件等通知.或者重启失败的job等.

这个异常就是发生在rerun的过程中.完整的异常如下:

 Unknown exception in doAs
java.lang.reflect.UndeclaredThrowableException: Unknown exception in doAs
	at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1127)
	at org.apache.oozie.service.KerberosHadoopAccessorService.createFileSystem(KerberosHadoopAccessorService.java:194)
	at org.apache.oozie.service.AuthorizationService.authorizeForApp(AuthorizationService.java:283)
	at org.apache.oozie.servlet.BaseJobServlet.checkAuthorizationForApp(BaseJobServlet.java:171)
	at org.apache.oozie.servlet.BaseJobServlet.doPut(BaseJobServlet.java:110)
	at javax.servlet.http.HttpServlet.service(HttpServlet.java:640)
	at org.apache.oozie.servlet.JsonRestServlet.service(JsonRestServlet.java:281)
	at javax.servlet.http.HttpServlet.service(HttpServlet.java:717)
	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:290)
	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
	at org.apache.oozie.servlet.AuthFilter$2.doFilter(AuthFilter.java:123)
	at com.cloudera.alfredo.server.AuthenticationFilter.doFilter(AuthenticationFilter.java:371)
	at org.apache.oozie.servlet.AuthFilter.doFilter(AuthFilter.java:128)
	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:235)
	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
	at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:233)
	at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:191)
	at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:127)
	at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:102)
	at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:109)
	at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:298)
	at org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:857)
	at org.apache.coyote.http11.Http11Protocol$Http11ConnectionHandler.process(Http11Protocol.java:588)
	at org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:489)
	at java.lang.Thread.run(Thread.java:662)
Caused by: java.security.PrivilegedActionException: org.apache.oozie.service.HadoopAccessorException: E0903: Configuration does not have Jobtracker Kerberos name or rule
	at java.security.AccessController.doPrivileged(Native Method)
	at javax.security.auth.Subject.doAs(Subject.java:396)
	at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1115)
	... 24 more
Caused by: org.apache.oozie.service.HadoopAccessorException: E0903: Configuration does not have Jobtracker Kerberos name or rule
	at org.apache.oozie.service.KerberosHadoopAccessorService.validateConf(KerberosHadoopAccessorService.java:244)
	at org.apache.oozie.service.KerberosHadoopAccessorService.access$000(KerberosHadoopAccessorService.java:49)
	at org.apache.oozie.service.KerberosHadoopAccessorService$3.run(KerberosHadoopAccessorService.java:202)
	at org.apache.oozie.service.KerberosHadoopAccessorService$3.run(KerberosHadoopAccessorService.java:194)
	... 27 more

 

    网上并没有找到解决方案,也不清楚 Jobtracker Kerberos到底是啥玩意.

于是跟踪下源码看,终于搞定.

    1.解决方案:

          只要在你的workflow/wf_job.properties,也就是你的workflow配置文件加上以下代码就行:

           mapreduce.jobtracker.kerberos.principal=mapred/_HOST@LOCALHOST

           dfs.namenode.kerberos.principal=hdfs/_HOST@LOCALHOST

    2.原理:

       跟踪源码发现,有如下判断之后才会触发该异常:

private void validateConf(Configuration conf) throws HadoopAccessorException {
        if (conf.get(WorkflowAppService.HADOOP_JT_KERBEROS_NAME) == null) {
            throw new HadoopAccessorException(ErrorCode.E0903);
        }
        if (conf.get(WorkflowAppService.HADOOP_NN_KERBEROS_NAME) == null) {
            throw new HadoopAccessorException(ErrorCode.E0904);
        }
    }

        从中可以发现只要Configuration中包含WorkflowAppService.HADOOP_JT_KERBEROS_NAME也就是

mapreduce.jobtracker.kerberos.principal和WorkflowAppService.HADOOP_NN_KERBEROS_NAME(dfs.namenode.kerberos.principal)即可.所以可以在配置文件,或者hadoop/conf/中配置.

 

更多技术文章、感悟、分享、勾搭，请用微信扫描:

评论

1 楼 zhenghangcx 2013-08-09  

你好，我看了你关于oozie的文章，正好我最进也在研究，方便留一个联系方式我么，qq邮箱都行。我的邮箱 [e-mail]，大牛向你学习