浏览 2823 次
精华帖 (0) :: 良好帖 (0) :: 新手帖 (0) :: 隐藏帖 (0)
|
|
---|---|
作者 | 正文 |
发表时间:2012-03-15
版本:hadoop-1.0.0,hbase-0.92.0 在195,196,197上安装hadoop集群 结果:195-》namenode 196-》jobtracker,datanode 197-> SecondaryNameNode,datanode 1, 建hadoop用户 groupadd hadoop useradd hadoop -g hadoop passwd hadoop mkdir -p /opt/hadoop mkdir -p /opt/data/hadoop1 chown hadoop:hadoop /opt/hadoop chown hadoop:hadoop /opt/data/hadoop1 2,修改hosts vi /etc/hosts 10.10.249.195 master 10.10.249.196 slaver1 10.10.249.197 slaver2 3,生成密钥对 su - hadoop ssh-keygen -t rsa cd /home/hadoop/.ssh cat id_rsa.pub > authorized_keys cd /home/hadoop/.ssh chmod 700 ../.ssh chmod 600 * chmod 644 authorized_keys 4,配置无密码 ssh登录 master: cd /home/hadoop/.ssh scp -r id_rsa.pub hadoop@10.10.249.196:~/.ssh/pubkey_master scp -r id_rsa.pub hadoop@10.10.249.197:~/.ssh/pubkey_master slaver1: cd /home/hadoop/.ssh cat pubkey_master >> authorized_keys scp -r id_rsa.pub hadoop@master:~/.ssh/pubkey_slaver1 scp -r id_rsa.pub hadoop@slaver2:~/.ssh/pubkey_slaver1 slaver2: cd /home/hadoop/.ssh cat pubkey_master >> authorized_keys cat pubkey_slaver1 >> authorized_keys scp -r id_rsa.pub hadoop@master:~/.ssh/pubkey_slaver2 master: cat pubkey_slaver1 >> authorized_keys cat pubkey_slaver2 >> authorized_keys # su root 修改 /etc/ssh/sshd_config PasswordAuthentication no 5:hadoop安装 conf/core-site.xml <?xml-stylesheet type="text/xsl" href="configuration.xsl"?> <configuration> <property> <name>fs.default.name</name> <value>hdfs://master:54310</value> </property> <property> <name>hadoop.tmp.dir</name> <value>/home/hadoop/hadoop_home/var</value> </property> </configuration> conf/mapred-site.xml <?xml version="1.0"?> <?xml-stylesheet type="text/xsl" href="configuration.xsl"?> <configuration> <property> <name>mapred.job.tracker</name> <value>slaver1:54311</value> </property> <property> <name>mapred.local.dir</name> <value>/home/hadoop/hadoop_home/var</value> </property> </configuration> conf/hdfs-site.xml <?xml version="1.0"?> <?xml-stylesheet type="text/xsl" href="configuration.xsl"?> <configuration> <property> <name>dfs.replication</name> <value>3</value> </property> </configuration> echo slaver2 > masters echo slaver1 > slaves echo slaver2 >> slaves 196,197同此配置 master: cd /opt/hadoop/hadoop-1.0.0/bin ./hadoop namenode -format start-all.sh ./bin/hadoop dfs -copyFromLocal conf/ /user/hadoop/bookdata ./bin/hadoop jar hadoop-examples-1.0.0.jar wordcount /user/hadoop/bookdata /user/hadoop/book_data_out job.tracker 50030 task.tracker 50060 dfs.http 50070 datanode.http 50075 secondary.http 50090 hadoop fs -mkdir /user/test hadoop fs -chown test:test /user/test hadoop dfsadmin -setSpaceQuota 1t /user/test hbase安装步骤 master:195 RegionServer:196,197 1,修改三台hadoop的配置hdfs-site.xml, <property> <name>dfs.support.append</name> <value>true</value> </property> <property> <name>dfs.datanode.max.xcievers</name> <value>4096</value> </property> restart hdfs #替换 hbase lib下的hadoop.jar为hdfs集群用的hadoop.jar 2,修改hbase-site.xml <configuration> <property> <name>hbase.rootdir</name> <value>hdfs://master:54310/hbase</value> <description>The directory shared by RegionServers. </description> </property> <property> <name>hbase.cluster.distributed</name> <value>true</value> <description>The mode the cluster will be in. Possible values are false: standalone and pseudo-distributed setups with managed Zookeeper true: fully-distributed with unmanaged Zookeeper Quorum (see hbase-env.sh) </description> </property> <property> <name>dfs.support.append</name> <value>true</value> </property> </configuration> 3,修改regionservers echo master > regionservers echo slaver1 >> regionservers echo slaver2 >> regionservers 4,修改hbase-env.sh : export HBASE_MANAGES_ZK=false #使用了单独安装的zookeeper, export JAVA_HOME=/export/servers/jdk1.6.0_25 5,将zookeeper集群的zoo.cfg 复制到 hbase的conf目录下。 6,复制配置好hbase到slaver1,slaver2 /opt/hadoop目录下 7,增加可以打开的文件句柄数,三台机器都执行。 su root echo -ne " * soft nofile 65536 * hard nofile 65536 " >>/etc/security/limits.conf 安装完成,启动 cd /opt/hadoop/hbase-0.92.0/bin ./start-hbase.sh master的web 端口,默认为60010,RegionServer web 端口 60030 声明:ITeye文章版权属于作者,受法律保护。没有作者书面许可不得转载。
推荐链接
|
|
返回顶楼 | |
发表时间:2013-03-05
楼主的配置是否可行?
|
|
返回顶楼 | |