[Hadoop] Hive

HIVE

http://hadoop.apache.org/hive/

http://wiki.apache.org/hadoop/Hive/GettingStarted

 

下载

wget http://labs.renren.com/apache-mirror/hadoop/hive/hive-0.5.0/hive-0.5.0-bin.tar.gz

 

Cloudera版本的Hadoop和Hive

http://archive.cloudera.com/cdh

 

安装步骤： 

hive-0.5.0-bin\README.txt

 

gzip -d hive-0.5.0-bin.tar.gz

tar xvf hive-0.5.0-bin.tar

cd hive-0.5.0-bin

chmod 700 bin/*

 

vi bin/hive-config.sh

export HIVE_HOME=/home/iic/hive-0.5.0-bin

export HADOOP_HOME=/home/iic/hadoop-0.20.2

export JAVA_HOME=/home/bmb/jdk1.6.0_16

 

 

bin/hadoop fs -mkdir       /tmp
bin/hadoop fs -mkdir       /user/hive/warehouse
bin/hadoop fs -chmod g+w   /tmp
bin/hadoop fs -chmod g+w   /user/hive/warehouse

 

 

启动

bin/hive

 

出错：

Exception in thread "main" java.lang.NoClassDefFoundError: org/apache/hadoop/hive/conf/HiveConf

原因：Hadoop的配置文件，被增加了hadoop-env.sh里面被增加了HADOOP_CLASSPATH的设置

原来是：

export HADOOP_CLASSPATH=$HBASE_HOME/hbase-0.20.3.jar:$HBASE_HOME/hbase-0.20.3-test.jar:$HBASE_HOME/conf:${HBASE_HOME}/lib/zookeeper-3.3.0.jar

修改成：

export HADOOP_CLASSPATH=$HADOOP_CLASSPATH:$HBASE_HOME/hbase-0.20.3.jar:$HBASE_HOME/hbase-0.20.3-test.jar:$HBASE_HOME/conf:${HBASE_HOME}/lib/zookeeper-3.3.0.jar

 

http://www.wangzhongyuan.com/archives/807.html

 

tail -f /tmp/iic/hive.log

 

Hive Web管理页面

bin/hive --service hwi

http://192.168.2.79:9999/hwi/

 

lib/hive_hwi.war 

 

异常处理

FAILED: Error in metadata: javax.jdo.JDOFatalDataStoreException: Failed to start database 'metastore_db', see the next exception for details.

 

原因，默认的“metastore_db”，同一时候，只能有一个进程对Hive进行操作，ps一下所有的hive,Kill掉他们。

 

  

批量删除进程

kill -9 `ps -ef |grep hive |awk '{print $2}' `

 

 

Oracle metastore

javax.jdo.option.ConnectionDriverName

javax.jdo.option.ConnectionURL

javax.jdo.option.ConnectionUserName

javax.jdo.option.ConnectionPassword

 

以下内容不能有空格，需要4个都替换

<property>

  <name>javax.jdo.option.ConnectionURL</name>

  <value>jdbc:oracle:thin:@192.168.2.82:1521:iic</value>

 </property>

<property>

  <name>javax.jdo.option.ConnectionDriverName</name>

  <value>oracle.jdbc.driver.OracleDriver</value>

</property>

<property>

  <name>javax.jdo.option.ConnectionUserName</name>

  <value>iic</value>

 </property>

<property>

  <name>javax.jdo.option.ConnectionPassword</name>

  <value>iic</value>

</property>

 

MetaStore相关表

\hive\src\metastore\src\model\package.jdo

修改jdo,可以修改相关的表

DBS

TYPE_FIELDS

TYPES

TYPE_FIELDS

TBLS

PARTITION_KEYS

TABLE_PARAMS

SERDES

SERDE_PARAMS

SORT_ORDER

SDS

COLUMNS

BUCKETING_COLS

SORT_COLS

SD_PARAMS

PARTITIONS

PARTITION_KEY_VALS

PARTITION_PARAMS