`

CDH4 HA test

 
阅读更多
场景:
      NN HA 设置成功,HA切换客户端出现异常,

错误分析
      用户执行Shell脚本问题
日志:

客户端
2012-08-01 14:37:07,798 WARN  ipc.Client (Client.java:run(787)) - Unexpected error reading responses on connection Thread[IPC Client (1333933549) connection to bigdata-3/172.16.206.206:9000 from peter,5,main]
java.lang.NullPointerException
at org.apache.hadoop.ipc.Client$Connection.receiveResponse(Client.java:852)
at org.apache.hadoop.ipc.Client$Connection.run(Client.java:781)
2012-08-01 14:37:07,807 WARN  retry.RetryInvocationHandler (RetryInvocationHandler.java:invoke(118)) - Exception while invoking complete of class ClientNamenodeProtocolTranslatorPB. Trying to fail over immediately.
2012-08-01 14:37:07,970 WARN  retry.RetryInvocationHandler (RetryInvocationHandler.java:invoke(118)) - Exception while invoking complete of class ClientNamenodeProtocolTranslatorPB after 1 fail over attempts. Trying to fail over after sleeping for 713ms.
2012-08-01 14:37:08,686 WARN  retry.RetryInvocationHandler (RetryInvocationHandler.java:invoke(118)) - Exception while invoking complete of class ClientNamenodeProtocolTranslatorPB after 2 fail over attempts. Trying to fail over after sleeping for 1596ms.
2012-08-01 14:37:10,286 WARN  retry.RetryInvocationHandler (RetryInvocationHandler.java:invoke(118)) - Exception while invoking complete of class ClientNamenodeProtocolTranslatorPB after 3 fail over attempts. Trying to fail over after sleeping for 2974ms.
2012-08-01 14:37:13,262 WARN  retry.RetryInvocationHandler (RetryInvocationHandler.java:invoke(118)) - Exception while invoking complete of class ClientNamenodeProtocolTranslatorPB after 4 fail over attempts. Trying to fail over after sleeping for 7861ms.

服务器端
2012-08-01 14:54:45,614 WARN org.apache.hadoop.security.UserGroupInformation: No groups available for user peter
2012-08-01 14:54:45,619 INFO org.apache.hadoop.hdfs.StateChange: BLOCK* NameSystem.allocateBlock: /user/peter/FS/100wan/1413. BP-283690147-172.16.206.206-1343792626658 blk_-6816230619303558443_3866{blockUCState=UNDER_CONSTRUCTION, primaryNodeIndex=-1, replicas=[ReplicaUnderConstruction[172.16.206.209:50010|RBW], ReplicaUnderConstruction[172.16.206.206:50010|RBW]]}
2012-08-01 14:54:46,529 INFO org.apache.hadoop.hdfs.StateChange: BLOCK* addStoredBlock: blockMap updated: 172.16.206.206:50010 is added to blk_-6816230619303558443_3866{blockUCState=UNDER_CONSTRUCTION, primaryNodeIndex=-1, replicas=[ReplicaUnderConstruction[172.16.206.209:50010|RBW], ReplicaUnderConstruction[172.16.206.206:50010|RBW]]} size 0
2012-08-01 14:54:46,529 INFO org.apache.hadoop.hdfs.StateChange: BLOCK* addStoredBlock: blockMap updated: 172.16.206.209:50010 is added to blk_-6816230619303558443_3866{blockUCState=UNDER_CONSTRUCTION, primaryNodeIndex=-1, replicas=[ReplicaUnderConstruction[172.16.206.209:50010|RBW], ReplicaUnderConstruction[172.16.206.206:50010|RBW]]} size 0
2012-08-01 14:54:46,531 INFO org.apache.hadoop.hdfs.StateChange: DIR* NameSystem.completeFile: file /user/peter/FS/100wan/1413 is closed by DFSClient_NONMAPREDUCE_-1368488343_1
2012-08-01 14:54:46,540 WARN org.apache.hadoop.security.ShellBasedUnixGroupsMapping: got exception trying to get groups for user peter
org.apache.hadoop.util.Shell$ExitCodeException: id: peter:无此用户

at org.apache.hadoop.util.Shell.runCommand(Shell.java:261)
at org.apache.hadoop.util.Shell.run(Shell.java:188)
at org.apache.hadoop.util.Shell$ShellCommandExecutor.execute(Shell.java:381)
at org.apache.hadoop.util.Shell.execCommand(Shell.java:467)
at org.apache.hadoop.util.Shell.execCommand(Shell.java:450)
at org.apache.hadoop.security.ShellBasedUnixGroupsMapping.getUnixGroups(ShellBasedUnixGroupsMapping.java:86)
at org.apache.hadoop.security.ShellBasedUnixGroupsMapping.getGroups(ShellBasedUnixGroupsMapping.java:55)
at org.apache.hadoop.security.Groups.getGroups(Groups.java:88)
at org.apache.hadoop.security.UserGroupInformation.getGroupNames(UserGroupInformation.java:1116)
at org.apache.hadoop.hdfs.server.namenode.FSPermissionChecker.<init>(FSPermissionChecker.java:51)
at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.checkPermission(FSNamesystem.java:4259)
at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.checkAncestorAccess(FSNamesystem.java:4236)
at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.startFileInternal(FSNamesystem.java:1579)
at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.startFile(FSNamesystem.java:1514)
at org.apache.hadoop.hdfs.server.namenode.NameNodeRpcServer.create(NameNodeRpcServer.java:408)
at org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolServerSideTranslatorPB.create(ClientNamenodeProtocolServerSideTranslatorPB.java:200)
at org.apache.hadoop.hdfs.protocol.proto.ClientNamenodeProtocolProtos$ClientNamenodeProtocol$2.callBlockingMethod(ClientNamenodeProtocolProtos.java:42590)
at org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine.java:427)
at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:916)
at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1692)
at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1688)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:396)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1232)
at org.apache.hadoop.ipc.Server$Handler.run(Server.java:1686)
分享到:
评论

相关推荐

    CDH HA部署

    ### CDH HA (High Availability) 部署详解 #### 一、概述 在大数据处理领域,Cloudera Distribution Including Apache Hadoop (CDH) 是一个广泛使用的平台,它不仅包含Hadoop的核心组件,还提供了丰富的附加服务。...

    cloudera CDH4 installation guide 4.0(pdf)

    在IT领域,特别是大数据处理平台的构建与维护中,Cloudera CDH4的安装与配置是一项关键技能。CDH4(Cloudera's Distribution Including Apache Hadoop)是Cloudera公司提供的一款基于Apache Hadoop的发行版,它集成...

    hadoop-cdh4-0.3.zip

    【标题】"hadoop-cdh4-0.3.zip" 提供的是针对Presto优化的CDH4 Hadoop发行版,这个版本是专为Presto大数据查询引擎设计的,旨在提高其在CDH4(Cloudera Distribution Including Apache Hadoop)环境中的性能和兼容性...

    presto-hive-cdh4-0.69.zip

    《Presto-Hive-CDH4:大数据SQL查询引擎的开源之旅》 Presto,这个由Facebook开源的大数据查询引擎,以其高效、灵活和可扩展性在大数据领域赢得了广泛的认可。尤其在处理PB级别的大规模数据时,Presto展现出了卓越...

    Hadoop CDH5.5.0 配置(Hdfs/Yarn HA)

    在CDH5.5.0中,HDFS(Hadoop Distributed File System)和YARN(Yet Another Resource Negotiator)是两个核心组件,它们在高可用性(HA)模式下的配置尤为重要。HDFS HA允许数据节点和名称节点的冗余,以确保即使单...

    Hadoop-2.0.0-cdh4.3.0安装手册(hbase-0.94.15-cdh4.6.0)

    本手册将详细阐述如何在 HA (High Availability,高可用性) 模式下安装 CDH4,并特别关注 HBase 的配置。 **1. 环境准备** - **主机**:首先,你需要准备至少三台机器作为集群的基础,包括一个 NameNode,一个 ...

    cdh hadoop官方安装文档

    3. **安装CDH4之前的准备工作(BEFORE YOU INSTALL CDH4 ON A CLUSTER)** 4. **支持的操作系统(SUPPORTED OPERATING SYSTEMS FOR CDH4)** 5. **CDH4安装流程(CDH4 INSTALLATION)** 6. **CDH4与MapReduce(CDH4 ...

    ubuntu-CDH6.3.2完整安装包网盘下载地址

    CDH6.3.2完整安装包网盘下载,包含 CDH-6.3.2-1.cdh6.3.2.p0.1605554-bionic.parcel、CDH-6.3.2-1.cdh6.3.2.p0.1605554-bionic.parcel.sha1、CDH-6.3.2-1.cdh6.3.2.p0.1605554-bionic.parcel.sha256、CDH-6.3.2-1....

    CDH6.3.2安装包下载

    由于CSDN上传文件大小限制,大家可以下载《CDH6.3.2下载.txt》获取网盘地址进行下载,我打包了CDH6.3.2 搭建所需要的各种安装文件,包括: manifest.json cloudera-manager.repo RPM-GPG-KEY-cloudera cm6.3.1-...

    cloudera CDH4 quick start guide 4.0(pdf)

    ### Cloudera CDH4快速入门指南 #### 关于本指南 Cloudera CDH4 快速入门指南是为那些希望快速上手并熟悉Cloudera Distribution Including Apache Hadoop (CDH4)的用户而设计的。此文档提供了安装、配置以及运行CDH...

    CDH4_High_Availability_Guide_b1.pdf

    CDH4 (Cloudera Distribution Including Apache Hadoop) 的高可用性 (High Availability, HA) 概念在 IT 行业中占据着重要的地位。尤其对于依赖于大数据处理的企业来说,确保 Hadoop 集群能够持续稳定运行是非常关键...

    CDH集群部署手册(For CDH5)

    为了提高数据的可靠性和系统的可用性,CDH5支持HDFS的高可用性(HA)模式。这通常包括配置NameNode HA,使用JournalNode进行日志同步,以及设置Quorum-based Storage策略。配置过程中需要关注Zookeeper的角色,以及...

    CDH7.1.5离线安装包下载

    CDH7及以上版本已经更名为CDP 本资源打包了CDH7.1.5 搭建所需要的各种安装文件,包括: cm7.2.4-redhat7.tar.gz manifest.json cloudera-manager.repo RPM-GPG-KEY-cloudera CDH-7.1.5-1.cdh7.1.5.p0.7431829-el7....

    CDH5.12.0资源下载

    CDH5.12.0

    cdh6.3.2 集成 Phoenix

    在大数据处理领域,CDH(Cloudera Distribution Including Apache Hadoop)是一款广泛使用的开源大数据平台,它包含了Hadoop生态系统中的多个组件,如HDFS、YARN、MapReduce、Hive等。而Phoenix是一个针对HBase的SQL...

    cdh7.1.7和cm7.4.4软件包下载及cdh7安装步骤

    cdh7.1.7包括: CDH-7.1.7-1.cdh7.1.7.p0.15945976-el7.parcel CDH-7.1.7-1.cdh7.1.7.p0.15945976-el7.parcel.sha1 CDH-7.1.7-1.cdh7.1.7.p0.15945976-el7.parcel.sha256 manifest.json cm7.4.7包括: cloudera-...

    CDH6.3.2安装包及下载地址

    ### CDH 6.3.2 安装包与下载指南 #### 一、背景介绍 自2021年1月31日起,Cloudera宣布其所有软件产品(包括Cloudera Data Hub,简称CDH)均需有效的订阅才能获取。这意味着用户无法再像以往那样免费下载CDH的各种...

    CDH之ZK升级.doc

    - 备份现有ZooKeeper Jar包:`cp /opt/cloudera/parcels/CDH-6.3.2-1.cdh6.3.2.p0.1605554/jars/zookeeper-3.4.5-cdh6.3.2.jar /opt/cloudera/parcels/CDH-6.3.2-1.cdh6.3.2.p0.1605554/jars/zookeeper-3.4.5-cdh...

    cdh6.3.2安装包免费下载

    cdh6.3.2安装包,第二个文件,因为cdh的parcel包很大, 所以分成了三个。 里边的文件包含: CDH-6.3.2-1.cdh6.3.2.p0.1605554-el7.parcel,1.93GB CDH-6.3.2-1.cdh6.3.2.p0.1605554-el7.parcel.sha,40字节 CDH-...

    hive-0.13.1-cdh5.3.6.rar

    hive-0.13.1-cdh5.3.6.rarhive-0.13.1-cdh5.3.6.rarhive-0.13.1-cdh5.3.6.rarhive-0.13.1-cdh5.3.6.rarhive-0.13.1-cdh5.3.6.rarhive-0.13.1-cdh5.3.6.rarhive-0.13.1-cdh5.3.6.rarhive-0.13.1-cdh5.3.6.rarhive-...

Global site tag (gtag.js) - Google Analytics