Hive + Hbase

在hive-site.xml里面有两个配置选项,配置好后可以应用hive执行到hbase集群
<property>
    <name>hive.aux.jars.path</name>
<value>file:///app/java/hive/lib/hive-hbase-handler-0.7.1.jar,file:///app/java/hive/lib/hbase-0.90.3.jar,file:///app/java/hive/lib/zookeeper-3.3.1.jar</value>
  </property>

<property>
    <name>hbase.zookeeper.quorum</name>
    <value>dfsname1.cptrue.com</value>
    <description>The directory shared by region servers.
    </description>
  </property>

Currently, we allocated mapred.child.java.opts  = -Xmx455037693, we need to increase to 1G. That would be tuning number 1. 

we need to increase the usage of mapred.job.reuse.jvm.num.tasks to -1. So every maps/reducers will reuse JVM instead of spanning the new ones.

用mapreduce jobtracker查看hive执行情况
http://jobtracker_server:50030/jobtracker.jsp  

hbase调优
http://hbase.apache.org/book/important_configurations.html#recommended_configurations

Hadoop的运行痕迹
http://www.cnblogs.com/forfuture1978/archive/2010/11/23/1884967.html


hive udaf 函数库 例子
http://blog.chinaunix.net/uid-9460004-id-3264684.html


http://www.cloudera.com/blog/2010/06/integrating-hive-and-hbase/

https://cwiki.apache.org/confluence/display/Hive/HBaseIntegration

https://cwiki.apache.org/confluence/display/Hive/StorageHandlers

https://cwiki.apache.org/confluence/display/Hive/GettingStarted

http://blog.csdn.net/hguisu/article/details/7282050

http://running.iteye.com/blog/898399

http://newitfarmer.com/bbs/

http://developer.yahoo.com/blogs/hadoop/posts/2010/08/pig_and_hive_at_yahoo/

http://www.cnblogs.com/OnlyXP/archive/2009/10/14/1583450.html

http://www.cnblogs.com/harrychinese/archive/2011/03/13/some_hadoop_hive_intro_articles.html

http://www.antlr.org/

https://issues.apache.org/jira/secure/attachment/12413737/hive_benchmark_2009-07-12.pdf

http://en.wikipedia.org/wiki/Directed_acyclic_graph

评论

1 楼 swzzm 2013-03-26  

.............文件加密了,密码了