impala集成LDAP

目的：
为解决kerberos安全机制下的impala，resin cache kerberos tgt maxrenewlife天失效问题。

说明：
impala启用LDAP后，会优先选择LDAP用户密码认证，当LDAP认证失败时自动选择kerberos安全认证。

步骤：
关闭防火墙，设置开机不启动防火墙
sudo /etc/init.d/iptables status
sudo /etc/init.d/iptables stop /  sudo service iptables stop
sudo chkconfig iptables off
安装LDAP
yum install db4 db4-utils db4-devel cyrus-sasl* krb5-server-ldap -y
yum install openldap openldap-servers openldap-clients openldap-devel compat-openldap -y
开启ldaps
/etc/sysconfig/ldap
SLAPD_LDAPS=yes
更新配置库
rm -rf /var/lib/ldap/*
cp /usr/share/openldap-servers/DB_CONFIG.example /var/lib/ldap/DB_CONFIG
chown -R ldap.ldap /var/lib/ldap
备份数据
cp -rf /etc/openldap/slapd.d /etc/openldap/slapd.d.bak
拷贝kerberos schema（可选择，当需要与kerberos结合使用时）
cp /usr/share/doc/krb5-server-ldap-1.10.3/kerberos.schema /etc/openldap/schema/
生成配置文件
touch /etc/openldap/slapd.conf
echo "include /etc/openldap/schema/corba.schema
include /etc/openldap/schema/core.schema
include /etc/openldap/schema/cosine.schema
include /etc/openldap/schema/duaconf.schema
include /etc/openldap/schema/dyngroup.schema
include /etc/openldap/schema/inetorgperson.schema
include /etc/openldap/schema/java.schema
include /etc/openldap/schema/misc.schema
include /etc/openldap/schema/nis.schema
include /etc/openldap/schema/openldap.schema
include /etc/openldap/schema/ppolicy.schema
include /etc/openldap/schema/collective.schema
include /etc/openldap/schema/kerberos.schema" > /etc/openldap/slapd.conf
echo -e "pidfile /var/run/openldap/slapd.pid\nargsfile /var/run/openldap/slapd.args" >> /etc/openldap/slapd.conf
红色字体可选择
更新slapd.d
slaptest -f /etc/openldap/slapd.conf -F /etc/openldap/slapd.d
chown -R ldap:ldap /etc/openldap/slapd.d && chmod -R 700 /etc/openldap/slapd.d
启动和设置开机启动
chkconfig --add slapd
chkconfig --level 345 slapd on
/etc/init.d/slapd start
验证
ps aux | grep slapd | grep -v grep
netstat -tunlp  | grep :389
失败时使用slapd -h ldap://127.0.0.1 -d 481查看日志
结合kerberos，如同上，可选择
kadmin.local -q "addprinc [e-mail]"
kadmin.local -q "addprinc -randkey ldap/[e-mail]"
kadmin.local -q "ktadd -k ldap.keytab ldap/[e-mail]"
拷贝到ldapserver机器
sudo chown ldap:ldap /etc/openldap/ldap.keytab && sudo  chmod 640 /etc/openldap/ldap.keytab
修改/etc/sysconfig/ldap
export KRB5_KTNAME=/etc/openldap/ldap.keytab

重启slapd
创建数据库
创建modify.ldif
dn: olcDatabase={2}bdb,cn=config
changetype: modify
replace: olcSuffix
olcSuffix: dc=yeahmobi,dc=com

dn: olcDatabase={2}bdb,cn=config
changetype: modify
replace: olcRootDN
# Temporary lines to allow initial setup
olcRootDN: uid=ldapadmin,ou=ndpmedia,dc=yeahmobi,dc=com

dn: olcDatabase={2}bdb,cn=config
changetype: modify
add: olcRootPW
olcRootPW: secret

dn: cn=config
changetype: modify
add: olcAuthzRegexp
olcAuthzRegexp: uid=([^,]*),cn=GSSAPI,cn=auth uid=$1,ou=ndpmedia,dc=yeahmobi,dc=com

dn: olcDatabase={2}bdb,cn=config
changetype: modify
add: olcAccess
# Everyone can read everything
olcAccess: {0}to dn.base="" by * read
# The ldapadm dn has full write access
olcAccess: {1}to * by dn="uid=ldapadmin,ou=ndpmedia,dc=yeahmobi,dc=com" write by * read

ldapmodify -Y EXTERNAL -H ldapi:/// -f modify.ldif

创建setup.ldif
dn: dc=yeahmobi,dc=com
objectClass: top
objectClass: dcObject
objectclass: organization
o: yeahmobi com
dc: yeahmobi

dn: ou=ndpmedia,dc=yeahmobi,dc=com
objectclass: organizationalUnit
ou: ndpmedia
description: Users

dn: ou=group,dc=yeahmobi,dc=com
objectClass: organizationalUnit
ou: group

dn: uid=ldapadmin,ou=ndpmedia,dc=yeahmobi,dc=com
objectClass: inetOrgPerson
objectClass: posixAccount
objectClass: shadowAccount
cn: LDAP admin account
uid: ldapadmin
sn: ldapadmin
uidNumber: 1001
gidNumber: 100
homeDirectory: /home/ldap
loginShell: /bin/bash

修改/etc/openldap/ldap.conf
BASE   dc=yeahmobi,dc=com
URI    ldap://ip-10-1-33-23.ec2.internal
运行
ldapadd -x -D "uid=ldapadmin,ou=ndpmedia,dc=yeahmobi,dc=com" -w secret -f setup.ldif

查询用户，修改密码，删除用户删除组
ldapsearch -LLL -x -D 'uid=ldapadmin,ou=ndpmedia,dc=yeahmobi,dc=com' -w secret -b 'dc=yeahmobi,dc=com' 'uid=hive'
ldappasswd -x -D 'uid=ldapadmin,ou=ndpmedia,dc=yeahmobi,dc=com' -w secret "uid=hive,ou=people,dc=yeahmobi,dc=com" -S
ldapdelete -x -w secret -D 'uid=ldapadmin,ou=ndpmedia,dc=yeahmobi,dc=com' "uid=hive,ou=ndpmedia,dc=yeahmobi,dc=com"
ldapdelete -x -w secret -D 'uid=ldapadmin,ou=ndpmedia,dc=yeahmobi,dc=com' "cn=hive,ou=group,dc=yeahmobi,dc=com"
使用migrationtools导入linux用户到LDAP参考参考文档
客户端配置
yum install openldap-clients -y
vim /etc/openldap/ldap.conf
BASE    dc=yeahmobi,dc=com
URI     ldap://ldaphost
Impala集成LDAP（使用CM配置）：
Service-wide->security
enable LDAP Authentication enable_ldap_auth true
LDAP URI ldap_uri填写ldap://ldap_server_hostname
Impala daemon group
advanced->impala daemon command line argument advanced configuration snippet填写
-ldap_baseDN=ou=ndpmedia,dc=yeahmobi,dc=com
-ldap_passwords_in_clear_ok=true(因为没有使用TLS加密，开启使用明文密码，不然daemon启动不来)
重启impala

JDBC使用：
private static final String CONNECTION_URL = "jdbc:hive2://" + IMPALAD_HOST + ':' + IMPALAD_JDBC_PORT + "/ym_system;user=impala;password=111111";
con = DriverManager.getConnection(CONNECTION_URL);
或者
private static final String CONNECTION_URL = "jdbc:hive2://" + IMPALAD_HOST + ':' + IMPALAD_JDBC_PORT + "/ym_system;";
con = DriverManager.getConnection(CONNECTION_URL,"impala","111111");
jar包使用5.0.2或者5.1.0版本的，使用最新5.2.0版本能连接成功，但不能指定db（使用默认default，而找不到表）。

Beenline使用：
          beeline -u "jdbc:hive2://impala_host:21050/default;" -n impala -p 111111

参考：
http://blog.javachen.com/2014/11/12/config-ldap-with-kerberos-in-cdh-hadoop/
按照如上配置或者参考资料中的配置，验证ldapsearch，会提示输入密码，并提示找不到database，经过测试属于正常想象，imapla可以正常访问。
https://wiki.debian.org/LDAP/LDAPUtils

评论

1 楼 linux91 2015-06-01  

版主：按你的步骤配置了，可是，执行 impala-shell -l -u test的时候，会报错：Error connecting: TTransportException, TSocket read 0 bytes ，不知道为什么ldap连不上，log里边是密码验证的错，不知道哪里需要调试，需要在cm里将hdfs的用户组映射改为ldap吗？