[2011-09-21] Hadoop and HBase 配置汇总

Hadoop

conf/masters

hadoop-master

conf/slaves

hadoop-master
hadoop-slave1
hadoop-slave2

conf/hadoop-env.sh

export JAVA_HOME=/home/michael/lib/jdk1.6.0_27

conf/hdfs-site.xml

<?xml version="1.0"?>
<?xml-stylesheet type="text/xsl" href="configuration.xsl"?>

<!-- Put site-specific property overrides in this file. -->

<configuration>
<property>
	<name>dfs.replication</name>
	<value>2</value>
</property>
</configuration>

conf/mapred-site.xml

<?xml version="1.0"?>
<?xml-stylesheet type="text/xsl" href="configuration.xsl"?>

<!-- Put site-specific property overrides in this file. -->

<configuration>
<property>
	<name>mapred.job.tracker</name>
	<value>hadoop-master:54311</value>
</property>
</configuration>

conf/core-site.xml

<?xml version="1.0"?>
<?xml-stylesheet type="text/xsl" href="configuration.xsl"?>

<!-- Put site-specific property overrides in this file. -->

<configuration>
<property>
	<name>fs.default.name</name>
	<value>hdfs://hadoop-master:54310</value>
</property>
</configuration>

HBase

conf/hbase-env.sh

export JAVA_HOME=/home/michael/lib/jdk1.6.0_27
export HBASE_MANAGES_ZK=true

conf/hbase-site.xml

<?xml version="1.0"?>
<?xml-stylesheet type="text/xsl" href="configuration.xsl"?>
<configuration>
<property>
	<name>hbase.rootdir</name>
	<value>hdfs://hadoop-master:54310/hbase</value>
</property>
<property>
	<name>hbase.cluster.distributed</name>
	<value>true</value>
</property>
<property>
	<name>hbase.master</name>
	<value>hdfs://hadoop-master:6000</value>
</property>

<property>
	<name>hbase.zookeeper.quorum</name>
	<value>yq-cl-svr3</value>
</property>
</configuration>

conf/regionservers

hadoop-master
hadoop-slave1
hadoop-slave2

将两个conf都配置好以后，复制到其他机器即可

update 2011-09-22:

为了让我们写的程序能够在hadoop上正常运行，需要将hbase的配置文件和CLASSPATH添加到hadoop的配置中

首先是配置文件

ln -s ../hbase-0.90.3-cdh3u1/conf/hbase-site.xml conf/hbase-site.xml

然后添加hbase的classpath到hadoop
conf/hadoop-env.sh

export JAVA_HOME=/home/michael/lib/jdk1.6.0_27
export HBASE_HOME=/home/michael/cdh/hbase-0.90.3-cdh3u1
export HADOOP_CLASSPATH=`${HBASE_HOME}/bin/hbase classpath`

很有用的Tip:
bin/hadoop classpath 和 bin/hbase classpath 可以查看相应的classpath