lookqlp
- 浏览: 345730 次
- 性别:
- 来自: 上海
-
文章分类
最新评论
-
tpxcer：
不开启时可以的，而且开启以后各种坑。。。。
hue beeswax权限管理 -
yangze：
博主请教一个问题，hue 控制hive表的权限怎么弄？ 怎么联 ...
cloudera新增用户权限配置 -
linux91：
楼主你好，我用CM配置LDAP用户组映射，进入impala时， ...
sentry配置 -
linux91：
版主：按你的步骤配置了，可是，执行 impala-shell ...
impala集成LDAP -
lookqlp：
super_a 写道你好！找不到表这个问题是如何解决的，可以描 ...
hcatalog读取hive数据并写入hive
配置hive gateway机器
Caused by: MetaException(message:Could not connect to meta store using any of the URIs provided. Most recent failure: org.apache.thrift.transport.TTransportException: GSS initiate failed
at org.apache.thrift.transport.TSaslTransport.sendAndThrowMessage(TSaslTransport.java:221)
at org.apache.thrift.transport.TSaslTransport.open(TSaslTransport.java:297)
at org.apache.thrift.transport.TSaslClientTransport.open(TSaslClientTransport.java:37)
at org.apache.hadoop.hive.thrift.client.TUGIAssumingTransport$1.run(TUGIAssumingTransport.java:52)
at org.apache.hadoop.hive.thrift.client.TUGIAssumingTransport$1.run(TUGIAssumingTransport.java:49)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1548)
at org.apache.hadoop.hive.thrift.client.TUGIAssumingTransport.open(TUGIAssumingTransport.java:49)
at org.apache.hadoop.hive.metastore.HiveMetaStoreClient.open(HiveMetaStoreClient.java:296)
at org.apache.hadoop.hive.metastore.HiveMetaStoreClient.<init>(HiveMetaStoreClient.java:169)
at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:57)
at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
at java.lang.reflect.Constructor.newInstance(Constructor.java:526)
at org.apache.hadoop.hive.metastore.MetaStoreUtils.newInstance(MetaStoreUtils.java:1161)
at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.<init>(RetryingMetaStoreClient.java:62)
at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.getProxy(RetryingMetaStoreClient.java:72)
at org.apache.hadoop.hive.ql.metadata.Hive.createMetaStoreClient(Hive.java:2407)
at org.apache.hadoop.hive.ql.metadata.Hive.getMSC(Hive.java:2418)
at org.apache.hadoop.hive.ql.metadata.Hive.getDatabase(Hive.java:1141)
at org.apache.hadoop.hive.ql.metadata.Hive.databaseExists(Hive.java:1130)
at org.apache.hadoop.hive.ql.exec.DDLTask.showTables(DDLTask.java:2250)
at org.apache.hadoop.hive.ql.exec.DDLTask.execute(DDLTask.java:334)
at org.apache.hadoop.hive.ql.exec.Task.executeTask(Task.java:151)
at org.apache.hadoop.hive.ql.exec.TaskRunner.runSequential(TaskRunner.java:65)
at org.apache.hadoop.hive.ql.Driver.launchTask(Driver.java:1485)
at org.apache.hadoop.hive.ql.Driver.execute(Driver.java:1263)
at org.apache.hadoop.hive.ql.Driver.runInternal(Driver.java:1091)
at org.apache.hadoop.hive.ql.Driver.run(Driver.java:931)
at org.apache.hadoop.hive.ql.Driver.run(Driver.java:921)
at org.apache.hadoop.hive.cli.CliDriver.processLocalCmd(CliDriver.java:268)
at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:220)
at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:422)
at org.apache.hadoop.hive.cli.CliDriver.executeDriver(CliDriver.java:790)
at org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:684)
at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:623)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at org.apache.hadoop.util.RunJar.main(RunJar.java:212)
)
at org.apache.hadoop.hive.metastore.HiveMetaStoreClient.open(HiveMetaStoreClient.java:342)
at org.apache.hadoop.hive.metastore.HiveMetaStoreClient.<init>(HiveMetaStoreClient.java:169)
... 31 more
增加hdfs gateway后
Failed with exception java.io.IOException:java.io.IOException: Can't get Master Kerberos principal for use as renewer
14/11/18 12:39:41 ERROR CliDriver: Failed with exception java.io.IOException:java.io.IOException: Can't get Master Kerberos principal for use as renewer
java.io.IOException: java.io.IOException: Can't get Master Kerberos principal for use as renewer
at org.apache.hadoop.hive.ql.exec.FetchOperator.getNextRow(FetchOperator.java:557)
at org.apache.hadoop.hive.ql.exec.FetchOperator.pushRow(FetchOperator.java:495)
at org.apache.hadoop.hive.ql.exec.FetchTask.fetch(FetchTask.java:139)
at org.apache.hadoop.hive.ql.Driver.getResults(Driver.java:1578)
at org.apache.hadoop.hive.cli.CliDriver.processLocalCmd(CliDriver.java:280)
at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:220)
at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:422)
at org.apache.hadoop.hive.cli.CliDriver.executeDriver(CliDriver.java:790)
at org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:684)
at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:623)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at org.apache.hadoop.util.RunJar.main(RunJar.java:212)
Caused by: java.io.IOException: Can't get Master Kerberos principal for use as renewer
at org.apache.hadoop.mapreduce.security.TokenCache.obtainTokensForNamenodesInternal(TokenCache.java:116)
at org.apache.hadoop.mapreduce.security.TokenCache.obtainTokensForNamenodesInternal(TokenCache.java:100)
at org.apache.hadoop.mapreduce.security.TokenCache.obtainTokensForNamenodes(TokenCache.java:80)
at org.apache.hadoop.mapred.FileInputFormat.listStatus(FileInputFormat.java:202)
at org.apache.hadoop.mapred.FileInputFormat.getSplits(FileInputFormat.java:270)
at org.apache.hadoop.hive.ql.exec.FetchOperator.getRecordReader(FetchOperator.java:386)
at org.apache.hadoop.hive.ql.exec.FetchOperator.getNextRow(FetchOperator.java:521)
增加yarn gateway后解决
Caused by: MetaException(message:Could not connect to meta store using any of the URIs provided. Most recent failure: org.apache.thrift.transport.TTransportException: GSS initiate failed
at org.apache.thrift.transport.TSaslTransport.sendAndThrowMessage(TSaslTransport.java:221)
at org.apache.thrift.transport.TSaslTransport.open(TSaslTransport.java:297)
at org.apache.thrift.transport.TSaslClientTransport.open(TSaslClientTransport.java:37)
at org.apache.hadoop.hive.thrift.client.TUGIAssumingTransport$1.run(TUGIAssumingTransport.java:52)
at org.apache.hadoop.hive.thrift.client.TUGIAssumingTransport$1.run(TUGIAssumingTransport.java:49)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1548)
at org.apache.hadoop.hive.thrift.client.TUGIAssumingTransport.open(TUGIAssumingTransport.java:49)
at org.apache.hadoop.hive.metastore.HiveMetaStoreClient.open(HiveMetaStoreClient.java:296)
at org.apache.hadoop.hive.metastore.HiveMetaStoreClient.<init>(HiveMetaStoreClient.java:169)
at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:57)
at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
at java.lang.reflect.Constructor.newInstance(Constructor.java:526)
at org.apache.hadoop.hive.metastore.MetaStoreUtils.newInstance(MetaStoreUtils.java:1161)
at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.<init>(RetryingMetaStoreClient.java:62)
at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.getProxy(RetryingMetaStoreClient.java:72)
at org.apache.hadoop.hive.ql.metadata.Hive.createMetaStoreClient(Hive.java:2407)
at org.apache.hadoop.hive.ql.metadata.Hive.getMSC(Hive.java:2418)
at org.apache.hadoop.hive.ql.metadata.Hive.getDatabase(Hive.java:1141)
at org.apache.hadoop.hive.ql.metadata.Hive.databaseExists(Hive.java:1130)
at org.apache.hadoop.hive.ql.exec.DDLTask.showTables(DDLTask.java:2250)
at org.apache.hadoop.hive.ql.exec.DDLTask.execute(DDLTask.java:334)
at org.apache.hadoop.hive.ql.exec.Task.executeTask(Task.java:151)
at org.apache.hadoop.hive.ql.exec.TaskRunner.runSequential(TaskRunner.java:65)
at org.apache.hadoop.hive.ql.Driver.launchTask(Driver.java:1485)
at org.apache.hadoop.hive.ql.Driver.execute(Driver.java:1263)
at org.apache.hadoop.hive.ql.Driver.runInternal(Driver.java:1091)
at org.apache.hadoop.hive.ql.Driver.run(Driver.java:931)
at org.apache.hadoop.hive.ql.Driver.run(Driver.java:921)
at org.apache.hadoop.hive.cli.CliDriver.processLocalCmd(CliDriver.java:268)
at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:220)
at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:422)
at org.apache.hadoop.hive.cli.CliDriver.executeDriver(CliDriver.java:790)
at org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:684)
at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:623)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at org.apache.hadoop.util.RunJar.main(RunJar.java:212)
)
at org.apache.hadoop.hive.metastore.HiveMetaStoreClient.open(HiveMetaStoreClient.java:342)
at org.apache.hadoop.hive.metastore.HiveMetaStoreClient.<init>(HiveMetaStoreClient.java:169)
... 31 more
增加hdfs gateway后
Failed with exception java.io.IOException:java.io.IOException: Can't get Master Kerberos principal for use as renewer
14/11/18 12:39:41 ERROR CliDriver: Failed with exception java.io.IOException:java.io.IOException: Can't get Master Kerberos principal for use as renewer
java.io.IOException: java.io.IOException: Can't get Master Kerberos principal for use as renewer
at org.apache.hadoop.hive.ql.exec.FetchOperator.getNextRow(FetchOperator.java:557)
at org.apache.hadoop.hive.ql.exec.FetchOperator.pushRow(FetchOperator.java:495)
at org.apache.hadoop.hive.ql.exec.FetchTask.fetch(FetchTask.java:139)
at org.apache.hadoop.hive.ql.Driver.getResults(Driver.java:1578)
at org.apache.hadoop.hive.cli.CliDriver.processLocalCmd(CliDriver.java:280)
at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:220)
at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:422)
at org.apache.hadoop.hive.cli.CliDriver.executeDriver(CliDriver.java:790)
at org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:684)
at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:623)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at org.apache.hadoop.util.RunJar.main(RunJar.java:212)
Caused by: java.io.IOException: Can't get Master Kerberos principal for use as renewer
at org.apache.hadoop.mapreduce.security.TokenCache.obtainTokensForNamenodesInternal(TokenCache.java:116)
at org.apache.hadoop.mapreduce.security.TokenCache.obtainTokensForNamenodesInternal(TokenCache.java:100)
at org.apache.hadoop.mapreduce.security.TokenCache.obtainTokensForNamenodes(TokenCache.java:80)
at org.apache.hadoop.mapred.FileInputFormat.listStatus(FileInputFormat.java:202)
at org.apache.hadoop.mapred.FileInputFormat.getSplits(FileInputFormat.java:270)
at org.apache.hadoop.hive.ql.exec.FetchOperator.getRecordReader(FetchOperator.java:386)
at org.apache.hadoop.hive.ql.exec.FetchOperator.getNextRow(FetchOperator.java:521)
增加yarn gateway后解决
分享到：
发表评论
-
hive相关元数据迁移（mysql）
2015-11-18 18:27 2568mysqldump -hhost -uroot -ppassw ... -
hive dynamic partitions insert java.lang.OutOfMemoryError: Java heap space
2015-10-26 18:03 3089动态分区问题，如果数据量大或者当动态分区大甚至只有十几个时 ... -
yarn NullPointerException
2015-03-26 17:03 1435yarn重启后，部分nm启动不了，报空指针问题 20 ... -
mapreduce mapper access security hbase
2015-03-17 14:42 1216环境： security cdh 5.2.0 secu ... -
sentry服务后，几个权限问题
2015-03-10 16:08 8968以账户bi为例 问题一：账户bi beeline ldap后 ... -
cloudera新增用户权限配置
2015-03-05 16:13 3957目标： 给各个业务组提供不同用户及用户组，并有限制的访问h ... -
hive集成LDAP
2015-02-13 10:09 9744cloudera manager hive- sevice ... -
impala HA
2014-12-11 17:36 1877目的： 为impala jdbc提供统一的接口，作用参照htt ... -
impala集成LDAP
2014-12-11 12:55 7108目的： 为解决kerberos安 ... -
security cdh mapreduce access hbase
2014-12-02 15:09 1072执行mapreduce的用户必须是可以访问hdfs相应目录和执 ... -
hcatalog读取hive数据并写入hive
2014-12-01 17:49 18118参考http://www.cloudera.com/cont ... -
CDH5.0.2升级至CDH5.2.0
2014-12-01 16:59 9857升级需求 1.为支持spark k ... -
hive并行导入数据锁问题
2014-08-26 10:46 2205hive同时load数据时，即使是不同分区，也会lock表， ... -
hive常用命令
2014-08-06 10:59 11915整理一下，用的时候照着贴就行了。 1.hive数据导出 ... -
hive执行时找不到文件
2014-03-03 12:57 7590hive计算时找不到文件 2014-03-03 11:13:3 ... -
hive hive.optimize.ppd=false导致map数量很大
2014-01-03 11:05 1865hive官方https://cwiki.apache.org/ ... -
hive结合hbase数据处理解决方案测评二（优化篇）
2012-12-06 09:41 6565接上一篇，对hbase参数进行优化，主要是调整与查询效率相关的 ... -
hive结合hbase数据处理解决方案测评
2012-12-03 17:19 5173前言 我们考虑为什么要将hive与hbase结合，出于hbas ... -
java.lang.ClassNotFoundException: org.apache.hadoop.hive.hbase.HBaseSplit
2012-12-03 11:19 5038hive关联hbase表，使用sele ... -
hive rank 测试
2012-11-30 11:18 3481前言 最近在做数据的去重，想到一种类似于关系型数据库rank函 ...
相关推荐
hive-on-spark客户端
SQuirrel SQL Client是一款流行的开源SQL查询工具，它允许用户通过一个图形用户界面（GUI）来连接到各种类型的数据库，包括Hive。Hive是一个基于Hadoop的数据仓库系统，常用于大数据处理和分析。本篇文章将详细讲解...
hiveclient.properties
"hive-configuration.zip"这个压缩包很可能包含了一个与前端开发相关的开源库，特别是针对Hive配置的。Hive通常指的是Apache Hive，这是一个数据仓库工具，允许用户使用SQL（HQL）查询、管理以及存储大量结构化数据...
【标题】"前端开源库-hive-configuration"指的是一个专门针对前端开发的开源库，它主要功能是提供一种蜂窝状的配置管理机制。这个库的设计理念是将配置信息组织得像蜂窝一样有序，方便开发者管理和使用。"Hive-...
Hive是Apache Hadoop生态系统中的一个数据仓库工具，它允许用户使用SQL方言（HQL）进行数据查询、分析和管理大规模分布式数据集。在Windows环境下，Hive的使用需要特定的配置和执行文件。以下是对标题和描述中涉及的...
在大数据处理领域，Hive是一个非常重要的工具，它提供了一个基于Hadoop的数据仓库基础设施，用于数据查询、分析和管理大规模数据集。本教程将详细讲解如何在Linux环境下安装Hive客户端，以便进行数据操作和分析。 ...
在IT行业中，数据库管理和分析是至关重要的任务，而Hive作为一个大数据处理的仓库系统，它提供了对结构化数据的查询和分析能力。当需要通过图形化的数据库管理工具，如DBeaver，与Hive进行交互时，就需要用到特定的...
hive metastore hive metastore是hive的元数据管理服务，实际应用中很多第三方框架需要访问metastore服务，如spark,impala等。同样hive metastore也提供了java接口。 使用 import org.apache.hadoop.hive.conf....
《PyPI上的Hive Metastore Client：大数据仓库的关键组件》 在大数据处理领域，Hadoop和Hive是两个至关重要的工具。Hadoop提供了一个分布式文件系统（HDFS），而Hive则是一个基于Hadoop的数据仓库系统，允许用户...
- `hadoop-client`: 包含Hadoop生态系统的客户端组件，如HDFS和YARN的客户端。 - `hive-exec`: Hive的执行引擎，包含查询解析、优化和执行的相关类。 - `hive-service`: 提供Hive Server 2等服务的组件。 - `...
"HIVE安装及详解" HIVE是一种基于Hadoop的数据仓库工具，主要用于处理和分析大规模数据。下面是关于HIVE的安装及详解。 HIVE基本概念 HIVE是什么？HIVE是一种数据仓库工具，主要用于处理和分析大规模数据。它将...
1. **Hive Client**：这是连接到Hive服务器并执行查询的客户端接口。它包含了HQL解析器、编译器和执行器，使得开发者可以通过编写Java代码或者使用命令行工具与Hive交互。 2. **Hive Metastore**：这部分jar包用于...
private HiveJdbcClient hiveClient; @GetMapping("/query") public List, Object>> queryData() { String sql = "SELECT * FROM my_table"; return hiveClient.query(sql); } } ``` 这个例子中，我们定义了...
该参数决定Hive是否应该自动地根据输入文件大小，在本地运行（在GateWay运行）。如果设置为true，则Hive会根据输入文件大小自动选择是否在本地运行，默认值为true。如果设置为false，则Hive将总是在线上模式运行，...
Used by metastore client to connect to remote metastore. <!-- 元数据使用mysql数据库 --> <name>javax.jdo.option.ConnectionURL <value>jdbc:mysql://bigdata01:3306/hive?createDatabaseIfNotExist=...
- `hadoop-client-*.jar`: Hadoop客户端库，用于与HDFS交互。 - `slf4j-api-*.jar`和`slf4j-log4j12-*.jar`: 日志框架，用于记录Hive操作的日志。 - `commons-lang3-*.jar`等其他依赖库：Hive运行时可能需要的...
本压缩包“hive远程连接工具和jar.zip”包含了用于远程连接Hive的重要组件和必需的JAR文件，主要涉及两个关键部分：SQuirreL SQL Client和Hive所需的JAR包。 首先，SQuirreL SQL Client是一个开源的、多数据库的、...
在大数据处理领域，Apache Hive是一个基于Hadoop的数据仓库工具，它允许用户使用SQL（HQL，Hive Query Language）查询存储在Hadoop集群中的大型数据集。Hive JDBC（Java Database Connectivity）是Hive提供的一种...
其中，`<hostname>`是Hive服务器的IP地址或域名，`<port>`是HiveServer2服务的监听端口，`<database>`是默认的Hive数据库，`<configuration parameters>`可选，用于设置Hive的配置参数。 4. 指定JDBC驱动：在...