hive和hbase的整合

使用hbase作为用户行为分析的数据库，为了提高效率，集成了hive。
安装
使用cloudera的cdh3u0版本，hbase安装见http://jinghong.iteye.com/blog/1051732，
解压hive到/home/hadoop/cdh3

tar zxvf hive-0.7.0-cdh3u0.tar.gz -C /home/hadoop/cdh3

在hive/conf目录下复制hive-default.xml到hive-site.xml
修改hive-site.xml，添加

<property>
  <name>hive.aux.jars.path</name>
  <value>file:///home/hadoop/cdh3/hive-0.7.0-cdh3u0/lib/hive-hbase-handler-0.7.0-cdh3u0.jar,file:///home/hadoop/cdh3/hive-0.7.0-cdh3u0/lib/hbase-0.90.1-cdh3u0.jar,file:///home/hadoop/cdh3/hive-0.7.0-cdh3u0/lib/zookeeper-3.3.1.jar</value>
</property>
<property>
  <name>hbase.zookeeper.quorum</name>
  <value>ubuntu-9,ubuntu-10,ubuntu-11</value>
  <description></description>
</property>

注意auxlib的写法，否则按service启动会有如下错误

java.io.IOException: Cannot create an instance of InputSplit class = org.apache.hadoop.hive.hbase.HBaseSplit:org.apache.hadoop.hive.hbase.HBaseSplit

修改hive-site.xml，修改meta源，用mysql作为保存meta的数据库

<property>
  <name>javax.jdo.option.ConnectionURL</name>
  <value>jdbc:mysql://192.168.0.53:3306/hive?useUnicode=true&amp;characterEncoding=UTF-8</value>
  <description>JDBC connect string for a JDBC metastore</description>
</property>
<property>
  <name>javax.jdo.option.ConnectionDriverName</name>
  <value>com.mysql.jdbc.Driver</value>
  <description>Driver class name for a JDBC metastore</description>
</property>
<property>
  <name>javax.jdo.option.ConnectionUserName</name>
  <value>hive</value>
  <description>username to use against metastore database</description>
</property>
<property>
  <name>javax.jdo.option.ConnectionPassword</name>
  <value>123456</value>
  <description>password to use against metastore database</description>
</property>

注意mysql的hive数据库字符集需要是latin1的。
修改/etc/profile

JAVA_HOME=/usr/local/jdk1.6.0_24
JRE_HOME=$JAVA_HOME/jre
CLASSPATH=.:$JAVA_HOME/lib:$JRE_HOME/lib:$CLASSPATH
HADOOP_HOME=/home/hadoop/cdh3/hadoop-0.20.2-cdh3u0
HBASE_HOME=/home/hadoop/cdh3/hbase-0.90.1-cdh3u0
HIVE_HOME=/home/hadoop/cdh3/hive-0.7.0-cdh3u0
PATH=$JAVA_HOME/bin:$HADOOP_HOME/bin:$HBASE_HOME/bin:$HIVE_HOME/bin:$PATH
export JAVA_HOME JRE_HOME CLASSPATH HADOOP_HOME HBASE_HOME HIVE_HOME PATH  

启动hive

hive

或按service启动

hive --service hiveserver