IntelliJ IDEA deploy hadoop

1.配置SSH

执行ssh keygen -t rsa

会在～/.ssh/id_rsa.pub 文件

将此文件通过scp远程拷贝到namenode节点

scp ~/.ssh/id_rsa.pub hadoop@namenode:~/.ssh/  

ssh hadoop@namenode  

将开发环境的id_rsa.pub文件拷贝到authorized_keys下

cat ~/.ssh/id_rsa.pub ~/.ssh/authorized_keys   

ssh无密码登陆已经完成->end.

2.Deploye.sh和Run.sh

#!/bin/sh  

echo "deploy jar"  

scp ../target/styhadoop-ch2-1.0.0-SNAPSHOT.jar hadoop@namenode:~/test/  

echo "deploy run.sh"  

scp run.sh hadoop@namenode:~/test/  

echo "change authority"  

ssh hadoop@namenode "chmod 755 ~/test/run.sh"  

echo "start run.sh"  

ssh hadoop@namenode "~/test/run.sh"

========================================================

#!/bin/sh  

echo "add jar to classpath"  

export HADOOP_CLASSPATH=~/test/styhadoop-ch2-1.0.0-SNAPSHOT.jar  

echo "run hadoop task"  

~/hadoop/bin/hadoop com.sweetop.styhadoop.MaxTemperature   input/  output/  

3.使用maven ant plugin发布

转载于:https://my.oschina.net/u/1442510/blog/307553