java远程连接standalone hbase -

fengyilin

浏览: 192288 次
性别:
来自: 南京

最近访客更多访客>>

hanfu_wei

jafeyyu

haydenzhu

map60148

博主相关

博客

微博

相册

留言

关于我

文章分类

社区版块

存档分类

java远程连接standalone hbase

博客分类：

Hadoop

hbase

本文主要描述如何实现在一台Linux机器上搭建一个standalone的hbase，在另外一台机器上通过API访问

服务器端环境搭建：
1.版本信息
java：1.8.0_172
hbase: 1.2.6

2.设置JAVA_HOME
在～/.bash_profile中追加如下内容

  JAVA_HOME=/usr/java/jdk1.8.0_172-amd64
  export JAVA_HOME

执行如下命令使上述设置立即生效

 source ～/.bash_profile

3.设置hostname

hostname docker05

4.下载hbase，点击下载
5.解压下载的文件

$ tar xzvf hbase-1.2.6-bin.tar.gz

6.修改conf下的文件，使能通过远程连接
6.1 进入conf文件夹

cd ./hbase-1.2.6/conf

6.2 修改hbase-site.xml文件，最主要是追加hbase.zookeeper.quorum属性，因为默认是localhost，只能在本机访问，修改成hostname才能远程访问

<?xml version="1.0"?>
<?xml-stylesheet type="text/xsl" href="configuration.xsl"?>
<!--
/**
 *
 * Licensed to the Apache Software Foundation (ASF) under one
 * or more contributor license agreements.  See the NOTICE file
 * distributed with this work for additional information
 * regarding copyright ownership.  The ASF licenses this file
 * to you under the Apache License, Version 2.0 (the
 * "License"); you may not use this file except in compliance
 * with the License.  You may obtain a copy of the License at
 *
 *     http://www.apache.org/licenses/LICENSE-2.0
 *
 * Unless required by applicable law or agreed to in writing, software
 * distributed under the License is distributed on an "AS IS" BASIS,
 * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
 * See the License for the specific language governing permissions and
 * limitations under the License.
 */
-->
<configuration>
 <property>
    <name>hbase.rootdir</name>
    <value>file:///root/data/hbase/data</value>
  </property>
  <property>
    <name>hbase.zookeeper.property.dataDir</name>
    <value>/root/data/hbase/zookeeper</value>
  </property>
   <property>
    <name>hbase.client.retries.number</name>
    <value>5</value>
  </property>
  <property>
    <name>hbase.zookeeper.quorum</name>
    <!--注意这里改为自己的hostname-->
    <value>docker05</value>
  </property>
   <property>
    <name>hbase.zookeeper.property.clientport</name>
    <value>2181</value>
  </property>
  <property>
    <name>hbase.unsafe.stream.capability.enforce</name>
    <value>false</value>
    <description>
      Controls whether HBase will check for stream capabilities (hflush/hsync).

      Disable this if you intend to run on LocalFileSystem, denoted by a rootdir
      with the 'file://' scheme, but be mindful of the NOTE below.

      WARNING: Setting this to false blinds you to potential data loss and
      inconsistent system state in the event of process and/or node failures. If
      HBase is complaining of an inability to use hsync or hflush it's most
      likely not a false positive.
    </description>
  </property>
</configuration>

6.3 修改regionservers文件
删除localhost，修改成hostname

7.启动hbase

cd ./hbase-1.2.6/bin

执行

./start-hbase.sh

正常情况下hbase就会启动好了，可通过jps命令查看是否有 **** HMaster 来判断

客户端配置
1.配置hostname
在客户端机器上也需要设置hostname，如192.168.0.172 docker05
2.将服务器上的hbase-site.xml 拷贝到客户端并放到工程根目录下
3.工程目录如下

4.工程的pom.xml文件

<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
	xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
	<modelVersion>4.0.0</modelVersion>
	<groupId>falcon.chengf</groupId>
	<artifactId>simple-hbase-test</artifactId>
	<version>0.0.1-SNAPSHOT</version>

	<dependencies>
		<dependency>
			<groupId>org.apache.hbase</groupId>
			<artifactId>hbase-client</artifactId>
			<version>1.3.1</version>
		</dependency>
	</dependencies>
</project>

5.测试类

/**
 * 
 */
package simple.hbase.test;

import java.io.IOException;
import java.io.InputStream;

import org.apache.hadoop.conf.Configuration;
import org.apache.hadoop.hbase.HBaseConfiguration;
import org.apache.hadoop.hbase.HColumnDescriptor;
import org.apache.hadoop.hbase.HTableDescriptor;
import org.apache.hadoop.hbase.TableName;
import org.apache.hadoop.hbase.client.Admin;
import org.apache.hadoop.hbase.client.Connection;
import org.apache.hadoop.hbase.client.ConnectionFactory;
import org.apache.hadoop.hbase.io.compress.Compression.Algorithm;

/**
 * @author: 作者： chengaofeng
 * @date: 创建时间：2018-05-31 16:12:35
 * @Description: TODO
 * @version V1.0
 */
public class HbaseTest {
	public static void main(String[] args) throws IOException {
		Configuration config = HBaseConfiguration.create();
		InputStream input = HbaseTest.class.getResourceAsStream("/hbase-site.xml");
		config.addResource(input);
		try (Connection connection = ConnectionFactory.createConnection(config); Admin admin = connection.getAdmin()) {

			HTableDescriptor table = new HTableDescriptor(TableName.valueOf("chengf"));
			table.addFamily(new HColumnDescriptor("columns").setCompressionType(Algorithm.NONE));

			System.out.print("Creating table. ");
			if (admin.tableExists(table.getTableName())) {
				admin.disableTable(table.getTableName());
				admin.deleteTable(table.getTableName());
			}
			admin.createTable(table);
			System.out.println(" create table ok.");
		}
	}
}

6.执行后，控制台信息如下

7.登录hbase服务器，可以看到chengf表创建成功

8.遇到的异常
8.1，只是设置了服务器的hostname，没有修改hbase-site.xml 和regionservers的内容，最终结果只能在服务器端通过shell操作hbase，在client机器上连接一直报org.apache.hadoop.hbase.ipc.FailedServerException: This server is in the failed servers list: localhost/127.0.0.1:60020 一类的错误，后来把hbase-site.xml 和regionservers改完后，重新启动才好了
8.2 启动报错

2019-12-06 16:16:21,277 ERROR [main] regionserver.HRegionServer: Failed construction RegionServer
java.lang.UnsupportedOperationException: Constructor threw an exception for org.apache.hadoop.hbase.ipc.NettyRpcServer
	at org.apache.hadoop.hbase.util.ReflectionUtils.instantiate(ReflectionUtils.java:66)
	at org.apache.hadoop.hbase.util.ReflectionUtils.instantiateWithCustomCtor(ReflectionUtils.java:45)
	at org.apache.hadoop.hbase.ipc.RpcServerFactory.createRpcServer(RpcServerFactory.java:66)
	at org.apache.hadoop.hbase.master.MasterRpcServices.createRpcServer(MasterRpcServices.java:368)
	at org.apache.hadoop.hbase.regionserver.RSRpcServices.<init>(RSRpcServices.java:1240)
	at org.apache.hadoop.hbase.regionserver.RSRpcServices.<init>(RSRpcServices.java:1196)
	at org.apache.hadoop.hbase.master.MasterRpcServices.<init>(MasterRpcServices.java:346)
	at org.apache.hadoop.hbase.master.HMaster.createRpcServices(HMaster.java:727)
	at org.apache.hadoop.hbase.regionserver.HRegionServer.<init>(HRegionServer.java:589)
	at org.apache.hadoop.hbase.master.HMaster.<init>(HMaster.java:495)
	at org.apache.hadoop.hbase.master.HMasterCommandLine$LocalHMaster.<init>(HMasterCommandLine.java:308)
	at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
	at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
	at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
	at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
	at org.apache.hadoop.hbase.util.JVMClusterUtil.createMasterThread(JVMClusterUtil.java:132)
	at org.apache.hadoop.hbase.LocalHBaseCluster.addMaster(LocalHBaseCluster.java:227)
	at org.apache.hadoop.hbase.LocalHBaseCluster.<init>(LocalHBaseCluster.java:174)
	at org.apache.hadoop.hbase.master.HMasterCommandLine.startMaster(HMasterCommandLine.java:229)
	at org.apache.hadoop.hbase.master.HMasterCommandLine.run(HMasterCommandLine.java:140)
	at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
	at org.apache.hadoop.hbase.util.ServerCommandLine.doMain(ServerCommandLine.java:149)
	at org.apache.hadoop.hbase.master.HMaster.main(HMaster.java:3123)
Caused by: java.lang.reflect.InvocationTargetException
	at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
	at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
	at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
	at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
	at org.apache.hadoop.hbase.util.ReflectionUtils.instantiate(ReflectionUtils.java:58)
	... 22 more
Caused by: java.net.BindException: Can't assign requested address
	at sun.nio.ch.Net.bind0(Native Method)
	at sun.nio.ch.Net.bind(Net.java:433)
	at sun.nio.ch.Net.bind(Net.java:425)
	at sun.nio.ch.ServerSocketChannelImpl.bind(ServerSocketChannelImpl.java:223)
	at org.apache.hbase.thirdparty.io.netty.channel.socket.nio.NioServerSocketChannel.doBind(NioServerSocketChannel.java:128)
	at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AbstractUnsafe.bind(AbstractChannel.java:558)
	at org.apache.hbase.thirdparty.io.netty.channel.DefaultChannelPipeline$HeadContext.bind(DefaultChannelPipeline.java:1283)
	at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeBind(AbstractChannelHandlerContext.java:501)
	at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.bind(AbstractChannelHandlerContext.java:486)
	at org.apache.hbase.thirdparty.io.netty.channel.DefaultChannelPipeline.bind(DefaultChannelPipeline.java:989)
	at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel.bind(AbstractChannel.java:254)
	at org.apache.hbase.thirdparty.io.netty.bootstrap.AbstractBootstrap$2.run(AbstractBootstrap.java:364)
	at org.apache.hbase.thirdparty.io.netty.util.concurrent.AbstractEventExecutor.safeExecute(AbstractEventExecutor.java:163)

因为自己的hosts文件中配置了其他的信息，启动有影响，修改hosts，注释掉无关的配置，追加hostname 和回环IP的映射

127.0.0.1	localhost
127.0.0.1	chengaofeng
#255.255.255.255	broadcasthost
#192.168.0.138 docker02
#192.168.0.173 config-server1 mcsas-mongo kafka1 redis zookeeper1 mongo-master mcsas-mongo elasticsearch es1