Linux下成功编译hadoop-eclipse-plugin-${version}.jar

转：http://chqz1987.blog.163.com/blog/static/51438311201301511853170/

   最近在学习hadoop，首先第一步学着编译了下hadoop-eclipse插件，中间遇到了不少问题，但最终还是解决了。现将编译过程记录下来，便于后面使用。
      这里是基于hadoop-1.0.4和eclipse Indigo版本。
      这里以HADOOP_HOME来表示您电脑上hadoop的安装目录。
     
1、编辑${HADOOP_HOME}/src/contrib/下的build-contrib.xml文件
添加{version}和{eclipse.home}属性：

<projectname="hadoopbuildcontrib"xmlns:ivy="antlib:org.apache.ivy.ant"> 

<!-- 这里定义了 version & eclipse.home -->  

<propertyname="version"value="1.0.4"/>  

<propertyname="eclipse.home"value="/home/chqz/systool/eclipse/eclipse"/>
 

<propertyname="name"value="${ant.project.name}"/>  

<propertyname="root"value="${basedir}"/>  

<propertyname="hadoop.root"location="${root}/../../../"/>

2、编辑${HADOOP_HOME}/src/contrib/eclipse-plugin/下的build.xml文件
(1) 添加hadoop-jars path，并同时加入到classpath中：

<!-- 这里添加了 hadoop-jars -->
<pathid="hadoop-jars">
<filesetdir="${hadoop.root}/">
<includename="hadoop-*.jar"/>
</fileset>
</path>

<!-- Override classpath to include Eclipse SDK jars -->
<pathid="classpath">
<pathelementlocation="${build.classes}"/>
<pathelementlocation="${hadoop.root}/build/classes"/>
<pathrefid="eclipse-sdk-jars"/>

<!-- 将 hadoop-jars 添加到这里 -->
<pathrefid="hadoop-jars"/>
</path>

(2) 设置includeantruntime=on，防止compile时报warning：

<targetname="compile"depends="init, ivy-retrieve-common"unless="skip.contrib">
<echomessage="contrib: ${name}"/>
<javac
encoding="${build.encoding}"
srcdir="${src.dir}"
includes="**/*.java"
destdir="${build.classes}"
debug="${javac.debug}"
deprecation="${javac.deprecation}"

<!-- 设置includeantruntime=on,防止compile报warning -->

  includeantruntime="on">
<classpathrefid="classpath"/>
</javac>
</target>

(3) 添加将要打包到plugin中的第三方包列表：

<!-- Override jar target to specify manifest -->
<targetname="jar"depends="compile"unless="skip.contrib">
<mkdirdir="${build.dir}/lib"/>

  <!-- 这里主要修改的是file中的值，注意路径一定要正确 -->
<copyfile="${hadoop.root}/hadoop-core-${version}.jar"

tofile="${build.dir}/lib/hadoop-core.jar"verbose="true"/>
<copyfile="${hadoop.root}/lib/commons-cli-1.2.jar"

todir="${build.dir}/lib"verbose="true"/>
<copyfile="${hadoop.root}/lib/commons-lang-2.4.jar"

todir="${build.dir}/lib"verbose="true"/>
<copyfile="${hadoop.root}/lib/commons-configuration-1.6.jar"

todir="${build.dir}/lib"verbose="true"/>
<copyfile="${hadoop.root}/lib/jackson-mapper-asl-1.8.8.jar"

todir="${build.dir}/lib"verbose="true"/>
<copyfile="${hadoop.root}/lib/jackson-core-asl-1.8.8.jar"

todir="${build.dir}/lib"verbose="true"/>
<copyfile="${hadoop.root}/lib/commons-httpclient-3.0.1.jar"

todir="${build.dir}/lib"verbose="true"/>

<jar
jarfile="${build.dir}/hadoop-${name}-${version}.jar"
manifest="${root}/META-INF/MANIFEST.MF">
<filesetdir="${build.dir}"includes="classes/ lib/"/>
<filesetdir="${root}"includes="resources/ plugin.xml"/>
</jar>
</target>

3、执行ant命令以生成hadoop-eclipse-plugin-${version}.jar包：
   进入到${HADOOP_HOME}/src/contrib/eclipse-plugin/目录下，
   然后执行ant命令，输出结果如下：

[chqz@chqz eclipse-plugin]$ ant
Buildfile:/home/chqz/plugin/hadoop/hadoop-1.0.4/src/contrib/eclipse-plugin/build.xml

check-contrib:

init:
[echo] contrib: eclipse-plugin

init-contrib:

ivy-download:
[get]Getting: http://repo2.maven.org/maven2/org/apache/ivy/ivy/2.1.0/ivy-2.1.0.jar
[get]To:/home/chqz/plugin/hadoop/hadoop-1.0.4/ivy/ivy-2.1.0.jar
[get]Not modified - so not downloaded

ivy-probe-antlib:

ivy-init-antlib:

ivy-init:
[ivy:configure]::Ivy2.1.0-20090925235825:: http://ant.apache.org/ivy/ ::
[ivy:configure]:: loading settings :: file =/home/chqz/plugin/hadoop/hadoop-1.0.4/ivy/ivysettings.xml

ivy-resolve-common:
[ivy:resolve]:: resolving dependencies :: org.apache.hadoop#eclipse-plugin;working@chqz
[ivy:resolve]  confs:[common]
[ivy:resolve]  found commons-logging#commons-logging;1.0.4 in maven2
[ivy:resolve]  found log4j#log4j;1.2.15 in maven2
[ivy:resolve]:: resolution report :: resolve 152ms:: artifacts dl 4ms
---------------------------------------------------------------------
||            modules            ||   artifacts   |
|       conf       | number| search|dwnlded|evicted|| number|dwnlded|
---------------------------------------------------------------------
|      common      |2|0|0|0||2|0|
---------------------------------------------------------------------

ivy-retrieve-common:
[ivy:retrieve]:: retrieving :: org.apache.hadoop#eclipse-plugin [sync]
[ivy:retrieve]  confs:[common]
[ivy:retrieve]0 artifacts copied,2 already retrieved (0kB/8ms)
[ivy:cachepath] DEPRECATED:'ivy.conf.file'is deprecated,use'ivy.settings.file' instead
[ivy:cachepath]:: loading settings :: file =/home/chqz/plugin/hadoop/hadoop-1.0.4/ivy/ivysettings.xml

compile:
[echo] contrib: eclipse-plugin

jar:

BUILD SUCCESSFUL
Total time:2 seconds

最后成功生成的hadoop-eclipse-plugin-${version}.jar在${HADOOP_HOME}/build/contrib/eclipse-plugin下。

drwxrwxr-x.3 chqz chqz 40961月1522:50 classes
drwxrwxr-x.2 chqz chqz 40961月1521:49 examples
-rw-rw-r--.1 chqz chqz 56251591月1523:03 hadoop-eclipse-plugin-1.0.4.jar
drwxrwxr-x.2 chqz chqz 40961月1523:03 lib
drwxrwxr-x.3 chqz chqz 40961月1521:49 system
drwxrwxr-x.3 chqz chqz 40961月1521:49 test