1、从官网上下载源代码,并解压;
2、下载maven2.3.1版本(2.0.3版本也执行过没问题)
3、到%hadoop-2.3.0-src%/hadoop-maven-plugins/目录下, 执行
mvn clean install
4、在有些公司或网络下,可能mvn install命令会执行失败,报maven的资源库无法访问,我碰到的是会自动连到report1那个上面,这个可以修改%MAVEN_HOME%/conf/settings.xml,修改<mirrors></mirrors>中的内容,如下:
<mirrors>
<mirror>
<id>mirrorId</id>
<mirrorOf>repositoryId</mirrorOf>
<name>Human Readable Name for this Mirror.</name>
<url>http://repo.maven.apache.org/maven2/</url>//这个url如果不可用,可以使用其它能用的替换
</mirror>
</mirrors>
5、如果有的公司上网是通过proxy,则需要继续修改同一个文件的<proxies></proxies>,例如:
<proxies>
<proxy>
<id>optional</id>
<active>true</active>
<protocol>http</protocol>
<username>proxyuser</username>
<password>proxypass</password>
<host>proxy.host.net</host>
<port>80</port>
<nonProxyHosts>local.net|some.host.com</nonProxyHosts>
</proxy>
</proxies>
6、到这里,基本上mvn clean install这个命令可以执行完了;然后回退到上一个目录,也就是cd %hadoop-2.3.0-src%/,执行:
mvn eclipse:eclipse -DskipTests
也即是跳过测试了
7、上面命令执行过程中,一般会出现一个问题:
[ERROR] Failed to execute goal org.apache.hadoop:hadoop-maven-plugins:2.3.0:protoc (compile-protoc) on project hadoop-common: org.apache.maven.plugin.MojoExecutionException: 'protoc --version' did not return a version -> [Help 1]
[ERROR]
[ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch.
[ERROR] Re-run Maven using the -X switch to enable full debug logging.
[ERROR][ERROR] For more information about the errors and possible solutions, please read the following articles:
[ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/MojoExecutionException
[ERROR]
[ERROR] After correcting the problems, you can resume the build with the command
[ERROR] mvn <goals> -rf :hadoop-common
8、上面这个错误,可以看出是由于没有找到proto引起的:org.apache.maven.plugin.MojoExecutionException: 'protoc --version' did not return a version -> [Help 1]
9、win7下,可以直接从网上下载protoc-2.5.0-win32.zip,win7 64位也可以执行,我用的就是64位操作系统。然后设置环境变量,进行加载即可,也可以用一个偷懒的方法:
把protoc.exe编译器,放入program files/protoc/自定的文件夹下,将这个目录追加入path环境变量。这样mvn就能调用protoc.exe,并且你自己也可以在cmd中随意执行。
10、重新执行mvn eclipse:eclipse -DskipTests
11、等到的最终执行完的结果:
[INFO] Apache Hadoop Main ................................ SUCCESS [ 1.222 s]
[INFO] Apache Hadoop Project POM ......................... SUCCESS [ 1.376 s]
[INFO] Apache Hadoop Annotations ......................... SUCCESS [ 0.263 s]
[INFO] Apache Hadoop Project Dist POM .................... SUCCESS [ 0.189 s]
[INFO] Apache Hadoop Assemblies .......................... SUCCESS [ 0.214 s]
[INFO] Apache Hadoop Maven Plugins ....................... SUCCESS [ 2.718 s]
[INFO] Apache Hadoop MiniKDC ............................. SUCCESS [ 2.666 s]
[INFO] Apache Hadoop Auth ................................ SUCCESS [ 1.948 s]
[INFO] Apache Hadoop Auth Examples ....................... SUCCESS [ 0.191 s]
[INFO] Apache Hadoop Common .............................. SUCCESS [ 5.259 s]
[INFO] Apache Hadoop NFS ................................. SUCCESS [01:15 min]
[INFO] Apache Hadoop Common Project ...................... SUCCESS [ 0.132 s]
[INFO] Apache Hadoop HDFS ................................ SUCCESS [01:55 min]
[INFO] Apache Hadoop HttpFS .............................. SUCCESS [ 25.091 s]
[INFO] Apache Hadoop HDFS BookKeeper Journal ............. SUCCESS [ 15.946 s]
[INFO] Apache Hadoop HDFS-NFS ............................ SUCCESS [ 4.452 s]
[INFO] Apache Hadoop HDFS Project ........................ SUCCESS [ 0.497 s]
[INFO] hadoop-yarn ....................................... SUCCESS [ 0.150 s]
[INFO] hadoop-yarn-api ................................... SUCCESS [ 2.251 s]
[INFO] hadoop-yarn-common ................................ SUCCESS [ 26.932 s]
[INFO] hadoop-yarn-server ................................ SUCCESS [ 0.143 s]
[INFO] hadoop-yarn-server-common ......................... SUCCESS [ 8.565 s]
[INFO] hadoop-yarn-server-nodemanager .................... SUCCESS [ 51.856 s]
[INFO] hadoop-yarn-server-web-proxy ...................... SUCCESS [ 2.283 s]
[INFO] hadoop-yarn-server-resourcemanager ................ SUCCESS [ 4.427 s]
[INFO] hadoop-yarn-server-tests .......................... SUCCESS [ 9.880 s]
[INFO] hadoop-yarn-client ................................ SUCCESS [ 2.852 s]
[INFO] hadoop-yarn-applications .......................... SUCCESS [ 0.112 s]
[INFO] hadoop-yarn-applications-distributedshell ......... SUCCESS [ 3.241 s]
[INFO] hadoop-yarn-applications-unmanaged-am-launcher .... SUCCESS [ 1.021 s]
[INFO] hadoop-yarn-site .................................. SUCCESS [ 0.099 s]
[INFO] hadoop-yarn-project ............................... SUCCESS [ 0.151 s]
[INFO] hadoop-mapreduce-client ........................... SUCCESS [ 0.263 s]
[INFO] hadoop-mapreduce-client-core ...................... SUCCESS [ 25.972 s]
[INFO] hadoop-mapreduce-client-common .................... SUCCESS [ 8.810 s]
[INFO] hadoop-mapreduce-client-shuffle ................... SUCCESS [ 1.110 s]
[INFO] hadoop-mapreduce-client-app ....................... SUCCESS [ 6.729 s]
[INFO] hadoop-mapreduce-client-hs ........................ SUCCESS [ 8.809 s]
[INFO] hadoop-mapreduce-client-jobclient ................. SUCCESS [ 10.474 s]
[INFO] hadoop-mapreduce-client-hs-plugins ................ SUCCESS [ 0.708 s]
[INFO] Apache Hadoop MapReduce Examples .................. SUCCESS [ 18.896 s]
[INFO] hadoop-mapreduce .................................. SUCCESS [ 0.199 s]
[INFO] Apache Hadoop MapReduce Streaming ................. SUCCESS [ 8.281 s]
[INFO] Apache Hadoop Distributed Copy .................... SUCCESS [03:25 min]
[INFO] Apache Hadoop Archives ............................ SUCCESS [ 2.256 s]
[INFO] Apache Hadoop Rumen ............................... SUCCESS [ 1.381 s]
[INFO] Apache Hadoop Gridmix ............................. SUCCESS [ 4.527 s]
[INFO] Apache Hadoop Data Join ........................... SUCCESS [ 0.865 s]
[INFO] Apache Hadoop Extras .............................. SUCCESS [ 0.690 s]
[INFO] Apache Hadoop Pipes ............................... SUCCESS [ 0.139 s]
[INFO] Apache Hadoop OpenStack support ................... SUCCESS [ 1.876 s]
[INFO] Apache Hadoop Client .............................. SUCCESS [ 2.828 s]
[INFO] Apache Hadoop Mini-Cluster ........................ SUCCESS [ 1.942 s]
[INFO] Apache Hadoop Scheduler Load Simulator ............ SUCCESS [ 14.240 s]
[INFO] Apache Hadoop Tools Dist .......................... SUCCESS [ 7.795 s]
[INFO] Apache Hadoop Tools ............................... SUCCESS [ 0.123 s]
[INFO] Apache Hadoop Distribution ........................ SUCCESS [ 0.520 s]
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 11:45 min
[INFO] Finished at: 2014-04-05T12:52:52+08:00
[INFO] Final Memory: 79M/307M
[INFO] ------------------------------------------------------------------------
D:\Download\bigdata\hadoop-2.3.0-src>
12、最后在Eclipse中按照以下流程导入源代码:“File” → “Import”→ “Existing Projects into Workspace”。(我用的是eclipse-java-kepler)
13,代码导入后,可能会出现一些java类不存在,例如:EchoResponseProto.java/EmptyResponseProto,主要是测试类,可以通过google解决。
分享到:
相关推荐
为了能够在Win 7 64位系统上成功编译Hadoop 2.7.3,首先需要进行一系列的准备工作: 1. **安装JDK**:确保已安装Java Development Kit(JDK)1.7或以上版本,并正确设置`JAVA_HOME`环境变量。需要注意的是,如果`...
Hadoop源代码eclipse编译教程汇编 Hadoop是一个基于分布式计算的开源框架,主要用于处理大规模数据。为了更好地理解Hadoop的工作机制和实现原理,深入了解Hadoop的源代码是非常必要的。本文将详细介绍如何使用...
Hadoop源代码eclipse编译指南.zip、Hadoop源代码eclipse编译指南.zipHadoop源代码eclipse编译指南.zipHadoop源代码eclipse编译指南.zipHadoop源代码eclipse编译指南.zip
本教程主要围绕"Hadoop基础"和"Hadoop源代码eclipse编译教程"展开,旨在帮助初学者深入理解Hadoop并掌握其实际操作技能。 首先,我们要了解Hadoop的基本概念。Hadoop是由Apache基金会开发的一个开源项目,基于Java...
标签再次确认了主题,聚焦于使用Eclipse进行Hadoop开发的学习材料,特别是源代码实例。这对于开发者来说是一个很好的起点,能够帮助他们快速理解和实践Hadoop的编程模型,包括Map函数和Reduce函数的实现。 【压缩...
hadoop2.3 native centos6.5 x64
总之,编译Apache Hadoop 2.4.0涉及的主要步骤包括安装JDK、Maven、FindBugs(可选)以及Protocol Buffers编译器,然后下载源代码并使用Maven进行编译。这个过程对于理解Hadoop的内部工作原理,或者定制自己的Hadoop...
你可以将这些编译结果部署到本地文件系统或模拟Hadoop集群(例如通过MiniCluster)进行测试。 此外,Hadoop 2.6.4在Windows上的编译还可能涉及对配置文件的修改,例如`hadoop-env.sh`,以适应Windows路径。同时,...
Hadoop源代码分析完整版.pdf
资源名称:大数据处理系统:Hadoop源代码情景分析内容简介:Hadoop是目前重要的一种开源的大数据处理平台,读懂Hadoop的源代码,深入理解其各种机理,对于掌握大数据处理的技术有着显而易见的重要性。 本书从大数据...
【标签】"hadoop 源代码"进一步强调了我们讨论的主题,对于开发者和研究者来说,理解Hadoop的源代码是深入学习Hadoop工作原理、优化性能或进行定制化开发的关键。 在【压缩包子文件的文件名称列表】中,我们看到...
本文将深入Hadoop的源代码,探讨其设计原理和实现机制,帮助开发者更好地理解和优化Hadoop系统。 Hadoop的源代码分析可以从以下几个关键点开始: 1. **HDFS架构**:HDFS是一个分布式文件系统,设计用于在廉价硬件...
5. **编译与测试**:运行Maven或Gradle的构建命令,如`mvn clean install`,这会编译源代码并执行单元测试,确保代码的功能正确性。 6. **部署与集成**:编译完成后,将编译好的二进制文件部署到Hadoop集群中,然后...
本资源"hadop实战源代码Java"聚焦于利用Java编程语言与Hadoop Distributed File System (HDFS)进行交互,实现文件的上传、下载以及删除等核心功能。这里我们将深入探讨这些知识点。 首先,了解Hadoop的基本架构是...
2. **获取源码**:从Apache官方网站下载Hadoop 2.7.2的源代码压缩包,并将其解压到Cygwin的适当目录。 3. **配置环境**:在Cygwin环境下,使用`./configure`命令对Hadoop进行配置。你需要确保配置时指向了正确的...