HiveConf
java class for current Hive configuration options
Metastore Conf
All the metadata for Hive tables and partitions are stored in Hive Metastore.
there are 3 different ways to setup metastore server using different Hive configurations:
An embedded metastore is mainly used for unit tests. Only one process can connect to the metastore at a time, so it is not really a practical solution but works well for unit tests.Derby is the default database for the embedded metastore.
In local metastore setup, each Hive Client will open a connection to the datastore and make SQL queries against it. The following config will set up a metastore in a MySQL server. Make sure that the server is accessible from the machines where Hive queries are executed since this is a local store. Also make sure the jdbc client library is in the classpath of Hive Client.
In remote metastore setup, all Hive Clients will make a connection to a metastore server which in turn queries the datastore (MySQL in this example) for metadata. Metastore server and client communicate using Thrift Protocol.
Starting with Hive 0.5.0, you can start a Thrift server by executing the following command:
#hive --service metastore
In my situation, I configure the remote metastore mode
/path/to/hive-0.12.0-bin/conf/hive-site.xml
<property>
<name>hive.metastore.uris</name>
<value>thrift://192.168.122.1:9083</value>
</property>
<property>
<name>javax.jdo.option.ConnectionURL</name>
<value>jdbc:mysql://192.168.122.1/hivedb?createDatabaseIfNotExist=true</value>
<description>JDBC connect string for a JDBC metastore</description>
</property>
<property>
<name>javax.jdo.option.ConnectionDriverName</name>
<value>com.mysql.jdbc.Driver</value>
<description>Driver class name for a JDBC metastore</description>
</property>
<property>
<name>javax.jdo.option.ConnectionUserName</name>
<value>hive</value>
<description>username to use against metastore database</description>
</property>
<property>
<name>javax.jdo.option.ConnectionPassword</name>
<value>hive123</value>
<description>password to use against metastore database</description>
</property>
<property>
<name>hive.metastore.warehouse.dir</name>
<value>/user/hive/warehouse</value>
<description>location of default database for the warehouse</description>
</property>
#hive --service metastore #hive --service metastore -p <port_num>
You should create user 'hive' and grant privileges to him in mysql datatbase;
If you are using MySQL as the datastore for metadata, put MySQL client libraries in HIVE_HOME/lib before starting Hive Client or HiveMetastore Server.
when run command #hive --service metastore
there are some errors :
MetaException(message:Version information not found in metastore
Solution:
#schematool -help
#schematool -dbType mysql -initSchema
Could not create ServerSocket on address 0.0.0.0/0.0.0.0:9083
Solution:
#netstat -tulpn |grep 9083
#kill -s 9 5776 //metastore daemon is running,so kill it with its pid 5776
when start hive CLI
hive>show databases;
eroor: Unable to instantiate org.apache.hadoop.hive.metastore.HiveMetaStoreClient
Solution:
#hive -hiveconf hive.root.logger=DEBUG,console //check detail logs
---------------------
Others:
<property>
<name>hive.server2.thrift.port</name>
<value>10000</value>
<description>Port number of HiveServer2 Thrift interface.
Can be overridden by setting $HIVE_SERVER2_THRIFT_PORT</description>
</property>
<property>
<name>hive.server2.thrift.bind.host</name>
<value>192.168.122.1</value>
<description>Bind host on which to run the HiveServer2 Thrift interface.
Can be overridden by setting $HIVE_SERVER2_THRIFT_BIND_HOST</description>
</property>
Note:Starting in Hive release 0.11.0, HCatalog is installed and configured with Hive. The HCatalog server is the same as the Hive metastore.
references:
https://cwiki.apache.org/confluence/display/Hive/AdminManual+MetastoreAdmin
http://www.cloudera.com/content/cloudera-content/cloudera-docs/CDH5/latest/CDH5-Installation-Guide/cdh5ig_hive_schema_tool.html
http://www.cloudera.com/content/cloudera-content/cloudera-docs/CDH4/4.2.0/CDH4-Installation-Guide/cdh4ig_topic_18_4.html?scroll=topic_18_4_1_unique_1__title_508_unique_1
相关推荐
1. 创建一个目录用于存放Derby数据库,例如:/var/lib/hive/metastore。 2. 修改Hive配置文件($HIVE_HOME/conf/hive-site.xml),添加以下内容: ```xml <configuration> <name>javax.jdo.option.ConnectionURL ...
其中,`hive.metastore.uris` 指定Hive元数据存储的位置,通常为Thrift服务器;`hive.metastore.warehouse.dir` 是HDFS上的仓库目录;`javax.jdo.option.ConnectionURL` 等配置是与MySQL数据库的连接信息,用于存储...
MySQL 是 Hive 的 metastore,用于存储 Hive 的元数据。安装 MySQL 的步骤如下: a) 使用 yum 安装 MySQL:`yum install mysql` b) 安装 MySQL-devel:`yum install mysql-devel` c) 安装 MariaDB:`yum install ...
logger.HiveMetaStore.name = org.apache.hadoop.hive.metastore.HiveMetaStore.audit logger.HiveMetaStore.level = INFO logger.HiveMetaStore.appenderRef.all.ref = DRFB ``` 5. **找到配置文件**: 默认...
- 启动Hive Metastore:`sbin/hive --service metastore` - 启动Hive Server2:`sbin/hive --service hiveserver2` 5. **验证Hive安装**: - 连接到Hive CLI:`bin/beeline -u jdbc:hive2://localhost:10000` ...
这个API包括Hive JDBC驱动和Hive Metastore服务客户端,使得Java应用程序能够与Hive Server交互。 3. **JDBC驱动**:Java Database Connectivity (JDBC) 是Java访问数据库的标准API。在Java中访问Hive,需要Hive的...
<name>hive.metastore.warehouse.dir <value>/user/hive/warehouse <!-- hive查询时输出列名 --> <name>hive.cli.print.header <value>true <!-- 显示当前数据库名 --> <name>hive.cli.print.current...
5.安装hive和mysq完成后,将mysql的连接jar包拷贝到$HIVE_HOME/lib目录下 如果出现没有权限的问题,在mysql授权(在安装mysql的机器上执行) mysql -uroot -p #(执行下面的语句 *.*:所有库下的所有表 %:任何IP地址...
2. **Metastore配置**:Metastore 是 Hive 中用于存储元数据的服务。可以选择内嵌模式(默认)、远程模式或者使用外部数据库(如 MySQL)。对于生产环境,建议使用外部数据库以提高稳定性和可维护性。 3. **...
删除 hive-site.xml 文件中标签<configuration>与</configuration>之间的所有内容,然后手动添加配置参数如下: <name>hive.metastore.warehouse.dir <value>/data/hive/warehouse <name>hive.metastore.local...
9. **commons-configuration-1.6.jar**:Apache Commons Configuration库,用于读取和管理配置文件,可能在Hive配置中使用。 10. **libthrift-0.9.3.jar**:Thrift库,是Facebook开源的一种跨语言的服务框架,Hive...
3. **配置 Hive**:在 Hive 的配置文件(如 `hcatalog-site.xml`)中,设置 `hive.metastore.pre.event.listeners` 属性,指定 Atlas 的 Hive Hook 类。 4. **重启服务**:更新配置后,重启 Atlas 和 Hive 服务以...
- 启动 Metastore 服务:`sbin/hive --service metastore` - 启动 HiveServer2 服务:`sbin/hive --service hiveserver2` 7. **验证安装**: - 运行 `hive` 命令进入交互模式,尝试执行简单的 SQL 查询来验证...
- 使用Derby存储方式时,运行Hive会在当前目录生成一个Derby文件和一个metastore_db目录。 - 在同一个目录下同时只能有一个Hive客户端能使用数据库,否则会出现错误。 #### 五、单用户模式(MySQL) ##### 安装...
<name>hive.metastore.schema.verification <value>false </configuration> ``` **6. 初始化Hive** 最后,初始化Hive元数据仓库: ```bash [hadoop@localhost bin]$ $HIVE_HOME/bin/schematool -dbType ...
- **Hive Metastore**:存储元数据的地方,如表结构、分区信息等,通常与Hive服务器分开部署。 - **Hadoop生态**:包括HDFS、YARN、MapReduce等,Hive是这个生态中的重要组成部分。 - **JDBC (Java Database ...
<value>jdbc:mysql://localhost:3306/hivemetastore?createDatabaseIfNotExist=true <name>javax.jdo.option.ConnectionDriverName <value>com.mysql.jdbc.Driver</value> <name>hive.metastore.uris ...
<name>hive.metastore.warehouse.dir <value>/user/hive/warehouse <description>location of default database for the warehouse </configuration> ``` 5. **添加MySQL驱动**:下载`mysql-connector-java-...
# Hive Configuration Directory can be controlled by: export HIVE_CONF_DIR=/home/fulong/Hive/apache-hive-0.13.1-bin/conf # Folder containing extra libraries required for hive compilation/execution ...