cdh5-centos7-安装

cdh官方网站已经收费，目前无法联机下载安装文件。文中所有文件可到网上搜索下载

 

一、以下操作需要master、worker1、worker2各自做一遍：

本次实战在所有机器上的操作，都使用root账号；

 

1 、hosts 主机名称设置

vi  /etc/hosts

 

192.168.8.44 cdh44

192.168.8.45 cdh45

192.168.8.46 cdh46

 

2、ssh免密登录

ssh-keygen -t rsa 

 

ssh root@cdh41

ssh root@cdh42

ssh root@cdh40

 

ssh-copy-id 192.168.8.40

ssh-copy-id 192.168.8.41

ssh-copy-id 192.168.8.42

 

 

3、jdk安装

#1、卸载原有jdk

rpm -qa|grep java

 

#1、安装jdk

mkdir -p /usr/lib/jvm

tar -zxvf  jdk-8u60-linux-x64.tar.gz  /usr/lib/jvm/

 

mv jdk1.8.0_60 /usr/lib/jvm/

 

#CDH默认的JDK目录是 /usr/java/default，因此要创建软链接，执行以下命令

mkdir /usr/java && ln -s /usr/lib/jvm/jdk1.8.0_60 /usr/java/default

 

vi /etc/profile

 

export JAVA_HOME=/usr/lib/jvm/jdk1.8.0_60

export JRE_HOME=${JAVA_HOME}/jre

export CLASSPATH=.:${JAVA_HOME}/lib:${JRE_HOME}/lib

export PATH=${JAVA_HOME}/bin:$PATH

 

source /etc/profile

 

4、mysql jar注意名称和路径，都不能错

mkdir -p  /usr/java

mkdir -p  /usr/share/java

cp mysql-connector-java.jar  /usr/java/mysql-connector-java.jar

 cp mysql-connector-java.jar  /usr/java/

 cp mysql-connector-java.jar  /usr/share/java/

5、关闭和禁用防火墙：

systemctl stop firewalld && systemctl disable firewalld

vi /etc/selinux/config

SELINUX=disabled

 

6、安装多个依赖包：

yum -y install chkconfig bind-utils psmisc libxslt zlib sqlite cyrus-sasl-plain cyrus-sasl-gssapi fuse fuse-libs redhat-lsb python portmap  httpd mod_ssl  wget

 

 

yum -y install httpd mod_ssl

yum install mod_ssl

 

7、配置 cm host

useradd --system --no-create-home --shell=/bin/false --comment "Cloudera SCM User" cloudera-scm

mkdir /opt/cloudera-manager

tar -zxvf cloudera-manager-centos7-cm5.16.1_x86_64.tar.gz -C /opt/cloudera-manager/

vi  /opt/cloudera-manager/cm-5.16.1/etc/cloudera-scm-agent/config.ini

server_host=master

 

8、mysql 设置

尝试使用docker创建mysql，但是最后进入cm节目安装时会提示错误，尽量使用如下方法安装mysql

 

mysql 安装

yum install mariadb-server

systemctl start mariadb && systemctl enable mariadb

mysql_secure_installation

mysql -u root -p123456

GRANT ALL PRIVILEGES ON *.* TO 'root'@'%' IDENTIFIED BY '123456' WITH GRANT OPTION;

flush privileges;

 

mysql 设置

创建访问用户“temp” 和分配“temp”用户权限

mysql -u root -p123456

create user 'temp'@'%' identified by 'temp';

grant all on *.* to 'temp'@'%' identified by 'temp' with grant option;

flush privileges;

 

分配“scm”用户权限

mysql> grant all privileges on *.* to'scm'@'%' identified by 'scm' with grant option;

mysql> flush privileges;

mysql> drop database cm;

mysql> show grants for scm;

mysql> use mysql;

mysql> select User,Host,Password from user;

 

二、SSH登录master机器，执行以下操作

 

1、创建parcel的本地仓库的文件夹，并且设置所有者和群组：

mkdir -p /opt/cloudera/parcel-repo

chown cloudera-scm:cloudera-scm  /opt/cloudera/parcel-repo

 

cp  CDH-5.16.1-1.cdh5.16.1.p0.3-el7.parcel   /opt/cloudera/parcel-repo/

cp  CDH-5.16.1-1.cdh5.16.1.p0.3-el7.parcel.sha    /opt/cloudera/parcel-repo/

cp   manifest.json    /opt/cloudera/parcel-repo/

 

 

2、初始化CM的mysql数据

cd /opt/cloudera-manager/cm-5.16.1/share/cmf/schema/

./scm_prepare_database.sh mysql mysqldb  -h mysqlserver  -umysqluser -pmysqlpwd --scm-host cmserverhost scm scm scm

./scm_prepare_database.sh mysql temp -h cdh41 -utemp -ptemp --scm-host cdh40 scm scm scm

 

 

三、worker机器上的设置

1、创建本地parcel缓存文件夹、给文件夹赋权限

mkdir -p /opt/cloudera/parcels

chown cloudera-scm:cloudera-scm /opt/cloudera/parcels

 

 

四、启动server(master机器)

cd /opt/cloudera-manager/cm-5.16.1/etc/init.d/

./cloudera-scm-server start

 

 

五、启动agent

cd /opt/cloudera-manager/cm-5.16.1/etc/init.d/

./cloudera-scm-agent start

 

python: error while loading shared libraries: libpython2.6.so.1.0: cannot open shared object file: No such file or directory

 

 

 

6、登录web页面

http://192.168.8.40:7180/   admin  admin

 

 

7、oozie安装web

 

下载ExtJS2.2

下载地址：http://archive.cloudera.com/gplextras/misc/ext-2.2.zip

上传到服务器并解压

[root@vm ~]# cd /opt/cloudera/parcels/CDH/lib/oozie/libext/

[root@vm libext]# unzip ext-2.2.zip 

[root@vm libext]# chown oozie:oozie -R ext-2.2

 

 8、安装时间同步工具：

 yum install -y ntpdate

时间同步：

ntpdate cn.ntp.org.cn