hive 三种启动方式及用途，关注通过jdbc连接的启动

 1， hive  命令行模式，直接输入/hive/bin/hive的执行程序，或者输入 hive –service cli

       用于linux平台命令行查询，查询语句基本跟mysql查询语句类似

 2， hive  web界面的启动方式，hive –service hwi  

       用于通过浏览器来访问hive，感觉没多大用途

 3， hive  远程服务 (端口号10000) 启动方式，./hive --service hiveserver >/dev/null 2>/dev/null &

 

       用java等程序实现通过jdbc等驱动的访问hive就用这种起动方式了，这个是程序员最需要的方式了

 

 

hive配置远程metastore

 

hive配置远程metastore的方法：

    1）首先配置hive使用本地mysql存储metastore（服务器A 111.121.21.23）（也可以使用远程mysql存储）

    2）配置完成后，在服务器A启动服务：bin/hive --service metastore  （默认监听端口是：9083）

    3）配置hive客户端，修改hive-site.xml：（服务器B-需要有hadoop环境）

        <property>

          <name>hive.metastore.local</name>

          <value>false</value>

          <description>controls whether to connect to remove metastore server or open a new metastore server in Hive Client JVM</description>

        </property>

        

        <property>

          <name>hive.metastore.uris</name>

          <value>thrift://111.121.21.23:9083</value>

          <description></description>

        </property>

     4）运行：bin/hive ,执行测试hql

 

     5）hive客户端连接成功后，hive服务器端输出以下日志：

Starting Hive Metastore Server

11/10/31 18:07:27 INFO metastore.HiveMetaStore: 0: Opening raw store with implemenation class:org.apache.hadoop.hive.metastore.ObjectStore

11/10/31 18:07:27 INFO metastore.ObjectStore: ObjectStore, initialize called

11/10/31 18:07:27 ERROR DataNucleus.Plugin: Bundle "org.eclipse.jdt.core" requires "org.eclipse.core.resources" but it cannot be resolved.

11/10/31 18:07:27 ERROR DataNucleus.Plugin: Bundle "org.eclipse.jdt.core" requires "org.eclipse.core.runtime" but it cannot be resolved.

11/10/31 18:07:27 ERROR DataNucleus.Plugin: Bundle "org.eclipse.jdt.core" requires "org.eclipse.text" but it cannot be resolved.

11/10/31 18:07:27 INFO DataNucleus.Persistence: Property datanucleus.cache.level2 unknown - will be ignored

11/10/31 18:07:27 INFO DataNucleus.Persistence: Property javax.jdo.option.NonTransactionalRead unknown - will be ignored

11/10/31 18:07:27 INFO DataNucleus.Persistence: ================= Persistence Configuration ===============

11/10/31 18:07:27 INFO DataNucleus.Persistence: DataNucleus Persistence Factory - Vendor: "DataNucleus"  Version: "2.0.3"

11/10/31 18:07:27 INFO DataNucleus.Persistence: DataNucleus Persistence Factory initialised for datastore URL="jdbc:mysql://localhost:3306/hive?createDatabaseIfNotExist=true" driver="com.mysql.jdbc.Driver" userName="hive"

11/10/31 18:07:27 INFO DataNucleus.Persistence: ===========================================================

11/10/31 18:07:28 INFO Datastore.Schema: Creating table `DELETEME1320055648261`

11/10/31 18:07:28 INFO Datastore.Schema: Schema Name could not be determined for this datastore

11/10/31 18:07:28 INFO Datastore.Schema: Dropping table `DELETEME1320055648261`

11/10/31 18:07:28 INFO Datastore.Schema: Initialising Catalog "hive", Schema "" using "None" auto-start option

11/10/31 18:07:28 INFO Datastore.Schema: Catalog "hive", Schema "" initialised - managing 0 classes

11/10/31 18:07:28 INFO metastore.ObjectStore: Setting MetaStore object pin classes with hive.metastore.cache.pinobjtypes="Table,StorageDescriptor,SerDeInfo,Partition,Database,Type,FieldSchema,Order"

11/10/31 18:07:28 INFO DataNucleus.MetaData: Registering listener for metadata initialisation

11/10/31 18:07:28 INFO metastore.ObjectStore: Initialized ObjectStore

11/10/31 18:07:28 WARN DataNucleus.MetaData: MetaData Parser encountered an error in file "jar:file:/opt/modules/hive/hive-0.7.1/lib/hive-metastore-0.7.1.jar!/package.jdo" at line 11, column 6 : cvc-elt.1: Cannot find the declaration of element 'jdo'. - Please check your specification of DTD and the validity of the MetaData XML that you have specified.

11/10/31 18:07:28 WARN DataNucleus.MetaData: MetaData Parser encountered an error in file "jar:file:/opt/modules/hive/hive-0.7.1/lib/hive-metastore-0.7.1.jar!/package.jdo" at line 312, column 13 : The content of element type "class" must match "(extension*,implements*,datastore-identity?,primary-key?,inheritance?,version?,join*,foreign-key*,index*,unique*,column*,field*,property*,query*,fetch-group*,extension*)". - Please check your specification of DTD and the validity of the MetaData XML that you have specified.

11/10/31 18:07:28 WARN DataNucleus.MetaData: MetaData Parser encountered an error in file "jar:file:/opt/modules/hive/hive-0.7.1/lib/hive-metastore-0.7.1.jar!/package.jdo" at line 359, column 13 : The content of element type "class" must match "(extension*,implements*,datastore-identity?,primary-key?,inheritance?,version?,join*,foreign-key*,index*,unique*,column*,field*,property*,query*,fetch-group*,extension*)". - Please check your specification of DTD and the validity of the MetaData XML that you have specified.

11/10/31 18:07:28 WARN DataNucleus.MetaData: MetaData Parser encountered an error in file "jar:file:/opt/modules/hive/hive-0.7.1/lib/hive-metastore-0.7.1.jar!/package.jdo" at line 381, column 13 : The content of element type "class" must match "(extension*,implements*,datastore-identity?,primary-key?,inheritance?,version?,join*,foreign-key*,index*,unique*,column*,field*,property*,query*,fetch-group*,extension*)". - Please check your specification of DTD and the validity of the MetaData XML that you have specified.

11/10/31 18:07:28 WARN DataNucleus.MetaData: MetaData Parser encountered an error in file "jar:file:/opt/modules/hive/hive-0.7.1/lib/hive-metastore-0.7.1.jar!/package.jdo" at line 416, column 13 : The content of element type "class" must match "(extension*,implements*,datastore-identity?,primary-key?,inheritance?,version?,join*,foreign-key*,index*,unique*,column*,field*,property*,query*,fetch-group*,extension*)". - Please check your specification of DTD and the validity of the MetaData XML that you have specified.

11/10/31 18:07:28 WARN DataNucleus.MetaData: MetaData Parser encountered an error in file "jar:file:/opt/modules/hive/hive-0.7.1/lib/hive-metastore-0.7.1.jar!/package.jdo" at line 453, column 13 : The content of element type "class" must match "(extension*,implements*,datastore-identity?,primary-key?,inheritance?,version?,join*,foreign-key*,index*,unique*,column*,field*,property*,query*,fetch-group*,extension*)". - Please check your specification of DTD and the validity of the MetaData XML that you have specified.

11/10/31 18:07:28 WARN DataNucleus.MetaData: MetaData Parser encountered an error in file "jar:file:/opt/modules/hive/hive-0.7.1/lib/hive-metastore-0.7.1.jar!/package.jdo" at line 494, column 13 : The content of element type "class" must match "(extension*,implements*,datastore-identity?,primary-key?,inheritance?,version?,join*,foreign-key*,index*,unique*,column*,field*,property*,query*,fetch-group*,extension*)". - Please check your specification of DTD and the validity of the MetaData XML that you have specified.

11/10/31 18:07:28 WARN DataNucleus.MetaData: MetaData Parser encountered an error in file "jar:file:/opt/modules/hive/hive-0.7.1/lib/hive-metastore-0.7.1.jar!/package.jdo" at line 535, column 13 : The content of element type "class" must match "(extension*,implements*,datastore-identity?,primary-key?,inheritance?,version?,join*,foreign-key*,index*,unique*,column*,field*,property*,query*,fetch-group*,extension*)". - Please check your specification of DTD and the validity of the MetaData XML that you have specified.

11/10/31 18:07:28 WARN DataNucleus.MetaData: MetaData Parser encountered an error in file "jar:file:/opt/modules/hive/hive-0.7.1/lib/hive-metastore-0.7.1.jar!/package.jdo" at line 576, column 13 : The content of element type "class" must match "(extension*,implements*,datastore-identity?,primary-key?,inheritance?,version?,join*,foreign-key*,index*,unique*,column*,field*,property*,query*,fetch-group*,extension*)". - Please check your specification of DTD and the validity of the MetaData XML that you have specified.

11/10/31 18:07:28 WARN DataNucleus.MetaData: MetaData Parser encountered an error in file "jar:file:/opt/modules/hive/hive-0.7.1/lib/hive-metastore-0.7.1.jar!/package.jdo" at line 621, column 13 : The content of element type "class" must match "(extension*,implements*,datastore-identity?,primary-key?,inheritance?,version?,join*,foreign-key*,index*,unique*,column*,field*,property*,query*,fetch-group*,extension*)". - Please check your specification of DTD and the validity of the MetaData XML that you have specified.

11/10/31 18:07:28 WARN DataNucleus.MetaData: MetaData Parser encountered an error in file "jar:file:/opt/modules/hive/hive-0.7.1/lib/hive-metastore-0.7.1.jar!/package.jdo" at line 666, column 13 : The content of element type "class" must match "(extension*,implements*,datastore-identity?,primary-key?,inheritance?,version?,join*,foreign-key*,index*,unique*,column*,field*,property*,query*,fetch-group*,extension*)". - Please check your specification of DTD and the validity of the MetaData XML that you have specified.

11/10/31 18:07:29 INFO DataNucleus.Persistence: Managing Persistence of Class : org.apache.hadoop.hive.metastore.model.MDatabase [Table : `DBS`, InheritanceStrategy : new-table]

11/10/31 18:07:29 INFO DataNucleus.Persistence: Managing Persistence of Field : org.apache.hadoop.hive.metastore.model.MDatabase.parameters [Table : `DATABASE_PARAMS`]

11/10/31 18:07:29 INFO Datastore.Schema: Validating 2 index(es) for table `DBS`

11/10/31 18:07:29 INFO Datastore.Schema: Validating 0 foreign key(s) for table `DBS`

11/10/31 18:07:29 INFO Datastore.Schema: Validating 2 unique key(s) for table `DBS`

11/10/31 18:07:29 INFO Datastore.Schema: Validating 2 index(es) for table `DATABASE_PARAMS`

11/10/31 18:07:29 INFO Datastore.Schema: Validating 1 foreign key(s) for table `DATABASE_PARAMS`

11/10/31 18:07:29 INFO Datastore.Schema: Validating 1 unique key(s) for table `DATABASE_PARAMS`

11/10/31 18:07:29 INFO DataNucleus.MetaData: Listener found initialisation for persistable class org.apache.hadoop.hive.metastore.model.MDatabase

11/10/31 18:07:29 INFO metastore.HiveMetaStore: Started the new metaserver on port [9083]...

11/10/31 18:07:29 INFO metastore.HiveMetaStore: Options.minWorkerThreads = 200

11/10/31 18:07:29 INFO metastore.HiveMetaStore: Options.maxWorkerThreads = 100000

11/10/31 18:07:29 INFO metastore.HiveMetaStore: TCP keepalive = true

11/10/31 18:09:10 INFO metastore.HiveMetaStore: 1: get_all_databases

11/10/31 18:09:10 INFO HiveMetaStore.audit: ugi=rsync   ip=/111.121.23.21      cmd=get_all_databases

11/10/31 18:09:10 INFO metastore.HiveMetaStore: 1: Opening raw store with implemenation class:org.apache.hadoop.hive.metastore.ObjectStore

11/10/31 18:09:10 INFO metastore.ObjectStore: ObjectStore, initialize called

11/10/31 18:09:10 INFO metastore.ObjectStore: Initialized ObjectStore

11/10/31 18:09:14 INFO metastore.HiveMetaStore: 1: get_database: vv

11/10/31 18:09:14 INFO HiveMetaStore.audit: ugi=rsync   ip=/111.121.23.21     cmd=get_database: vv

11/10/31 18:09:14 INFO metastore.HiveMetaStore: 1: get_database: vv

11/10/31 18:09:14 INFO HiveMetaStore.audit: ugi=rsync   ip=/111.121.23.21      cmd=get_database: vv

11/10/31 18:09:24 INFO metastore.HiveMetaStore: 1: get_table : db=vv tbl=vv_20111031

11/10/31 18:09:24 INFO HiveMetaStore.audit: ugi=rsync   ip=/111.121.23.21      cmd=get_table : db=vv tbl=vv_20111031

11/10/31 18:09:24 INFO DataNucleus.Datastore: The class "org.apache.hadoop.hive.metastore.model.MFieldSchema" is tagged as "embedded-only" so does not have its own datastore table.

11/10/31 18:09:24 INFO DataNucleus.Persistence: Managing Persistence of Class : org.apache.hadoop.hive.metastore.model.MSerDeInfo [Table : `SERDES`, InheritanceStrategy : new-table]

11/10/31 18:09:24 INFO DataNucleus.Datastore: The class "org.apache.hadoop.hive.metastore.model.MOrder" is tagged as "embedded-only" so does not have its own datastore table.

11/10/31 18:09:24 INFO DataNucleus.Persistence: Managing Persistence of Class : org.apache.hadoop.hive.metastore.model.MStorageDescriptor [Table : `SDS`, InheritanceStrategy : new-table]

11/10/31 18:09:24 INFO DataNucleus.Persistence: Managing Persistence of Class : org.apache.hadoop.hive.metastore.model.MTable [Table : `TBLS`, InheritanceStrategy : new-table]

11/10/31 18:09:24 INFO DataNucleus.Persistence: Managing Persistence of Field : org.apache.hadoop.hive.metastore.model.MSerDeInfo.parameters [Table : `SERDE_PARAMS`]

11/10/31 18:09:24 INFO DataNucleus.Persistence: Managing Persistence of Field : org.apache.hadoop.hive.metastore.model.MTable.parameters [Table : `TABLE_PARAMS`]

11/10/31 18:09:24 INFO DataNucleus.Persistence: Managing Persistence of Field : org.apache.hadoop.hive.metastore.model.MTable.partitionKeys [Table : `PARTITION_KEYS`]

11/10/31 18:09:24 INFO DataNucleus.Persistence: Managing Persistence of Field : org.apache.hadoop.hive.metastore.model.MStorageDescriptor.bucketCols [Table : `BUCKETING_COLS`]

11/10/31 18:09:24 INFO DataNucleus.Persistence: Managing Persistence of Field : org.apache.hadoop.hive.metastore.model.MStorageDescriptor.cols [Table : `COLUMNS`]

11/10/31 18:09:24 INFO DataNucleus.Persistence: Managing Persistence of Field : org.apache.hadoop.hive.metastore.model.MStorageDescriptor.parameters [Table : `SD_PARAMS`]

11/10/31 18:09:24 INFO DataNucleus.Persistence: Managing Persistence of Field : org.apache.hadoop.hive.metastore.model.MStorageDescriptor.sortCols [Table : `SORT_COLS`]

11/10/31 18:09:24 INFO Datastore.Schema: Validating 1 index(es) for table `SERDES`

11/10/31 18:09:24 INFO Datastore.Schema: Validating 0 foreign key(s) for table `SERDES`

11/10/31 18:09:24 INFO Datastore.Schema: Validating 1 unique key(s) for table `SERDES`

11/10/31 18:09:24 INFO Datastore.Schema: Validating 4 index(es) for table `TBLS`

11/10/31 18:09:24 INFO Datastore.Schema: Validating 2 foreign key(s) for table `TBLS`

11/10/31 18:09:24 INFO Datastore.Schema: Validating 2 unique key(s) for table `TBLS`

11/10/31 18:09:24 INFO Datastore.Schema: Validating 2 index(es) for table `SDS`

11/10/31 18:09:24 INFO Datastore.Schema: Validating 1 foreign key(s) for table `SDS`

11/10/31 18:09:24 INFO Datastore.Schema: Validating 1 unique key(s) for table `SDS`

11/10/31 18:09:24 INFO Datastore.Schema: Validating 2 index(es) for table `COLUMNS`

11/10/31 18:09:24 INFO Datastore.Schema: Validating 1 foreign key(s) for table `COLUMNS`

11/10/31 18:09:24 INFO Datastore.Schema: Validating 1 unique key(s) for table `COLUMNS`

11/10/31 18:09:24 INFO Datastore.Schema: Validating 2 index(es) for table `SERDE_PARAMS`

11/10/31 18:09:24 INFO Datastore.Schema: Validating 1 foreign key(s) for table `SERDE_PARAMS`

11/10/31 18:09:24 INFO Datastore.Schema: Validating 1 unique key(s) for table `SERDE_PARAMS`

11/10/31 18:09:24 INFO Datastore.Schema: Validating 2 index(es) for table `SD_PARAMS`

11/10/31 18:09:24 INFO Datastore.Schema: Validating 1 foreign key(s) for table `SD_PARAMS`

11/10/31 18:09:24 INFO Datastore.Schema: Validating 1 unique key(s) for table `SD_PARAMS`

11/10/31 18:09:24 INFO Datastore.Schema: Validating 2 index(es) for table `BUCKETING_COLS`

11/10/31 18:09:24 INFO Datastore.Schema: Validating 1 foreign key(s) for table `BUCKETING_COLS`

11/10/31 18:09:24 INFO Datastore.Schema: Validating 1 unique key(s) for table `BUCKETING_COLS`

11/10/31 18:09:24 INFO Datastore.Schema: Validating 2 index(es) for table `PARTITION_KEYS`

11/10/31 18:09:24 INFO Datastore.Schema: Validating 1 foreign key(s) for table `PARTITION_KEYS`

11/10/31 18:09:24 INFO Datastore.Schema: Validating 1 unique key(s) for table `PARTITION_KEYS`

11/10/31 18:09:24 INFO Datastore.Schema: Validating 2 index(es) for table `TABLE_PARAMS`

11/10/31 18:09:24 INFO Datastore.Schema: Validating 1 foreign key(s) for table `TABLE_PARAMS`

11/10/31 18:09:24 INFO Datastore.Schema: Validating 1 unique key(s) for table `TABLE_PARAMS`

11/10/31 18:09:24 INFO Datastore.Schema: Validating 2 index(es) for table `SORT_COLS`

11/10/31 18:09:24 INFO Datastore.Schema: Validating 1 foreign key(s) for table `SORT_COLS`

11/10/31 18:09:24 INFO Datastore.Schema: Validating 1 unique key(s) for table `SORT_COLS`

11/10/31 18:09:24 INFO DataNucleus.MetaData: Listener found initialisation for persistable class org.apache.hadoop.hive.metastore.model.MSerDeInfo

11/10/31 18:09:24 INFO DataNucleus.MetaData: Listener found initialisation for persistable class org.apache.hadoop.hive.metastore.model.MStorageDescriptor

11/10/31 18:09:24 INFO DataNucleus.MetaData: Listener found initialisation for persistable class org.apache.hadoop.hive.metastore.model.MTable

11/10/31 18:09:24 INFO DataNucleus.MetaData: Listener found initialisation for persistable class org.apache.hadoop.hive.metastore.model.MFieldSchema

11/10/31 18:10:10 INFO metastore.HiveMetaStore: 2: get_database: vv

11/10/31 18:10:10 INFO HiveMetaStore.audit: ugi=rsync   ip=/111.121.23.21     cmd=get_database: vv

11/10/31 18:10:10 INFO metastore.HiveMetaStore: 2: Opening raw store with implemenation class:org.apache.hadoop.hive.metastore.ObjectStore

11/10/31 18:10:10 INFO metastore.ObjectStore: ObjectStore, initialize called

11/10/31 18:10:10 INFO metastore.ObjectStore: Initialized ObjectStore

11/10/31 18:10:10 INFO metastore.HiveMetaStore: 2: get_database: vv

11/10/31 18:10:10 INFO HiveMetaStore.audit: ugi=rsync   ip=/111.121.23.21      cmd=get_database: vv

11/10/31 18:10:16 INFO metastore.HiveMetaStore: 2: get_table : db=vv tbl=vv_20111031

 

11/10/31 18:10:16 INFO HiveMetaStore.audit: ugi=rsync   ip=/111.121.23.21      cmd=get_table : db=vv tbl=vv_20111031