`
kavy
  • 浏览: 891070 次
  • 性别: Icon_minigender_1
  • 来自: 上海
社区版块
存档分类
最新评论

HOWTO install Hadoop on Windows

 
阅读更多

Installing the Hortonworks Data Platform for Windows couldn’t be easier. Lets take a look at how to install a one node cluster on your Windows Server 2012 machine. to let us know if you’d like more content like this.

 

msi_download

To start, download the HDP for Windows MSI at http://hortonworks.com/thankyou-hdp11-win/. It is about 460MB, and will take a moment to download. Documentation for the download is available here.

 

As indicated in the documentation here, first we must install Microsoft Visual C++ 2010 Redistributable Package (x64), available here.

Download and install .NET from here if you haven’t already.

We need to setup Java, which you can get here. We need to setup JAVA_HOME, which Hadoop requires. Make sure to install Java to somewhere without a space in the path, “Program Files” will not work!

To setup JAVA_HOME, in the file browsers -> right click computer -> Properties -> Advanced System Settings -> Environment variables. Then setup a new System variable called JAVA_HOME that points to your Java install (in this case, C:\java\jdk1.6.0_31).

JAVA_HOME

Finally, we need to download python from here and set the Path environment variable as we did JAVA_HOME. Go to Computer -> Properties -> Advanced System Settings -> Environment variables. Then append the install path to Python, for example C:\Python27, to this path after a ‘;’.

python_path

Verify your path is setup by entering a new shell and typing: python, which should run the python interpreter. Type quit() to exit. Now we’re ready for our configuration.

Next, notepad the file clusterproperties.txt, which we will setup for a simple, one node cluster operation. Note: first we need to discover our hostname, and enter it into our config instead of something generic like ‘localhost.’ Use the hostname command, for example:

hostname
WIN-4VLBRQK8FA8

We then place this hostname in our config. Be sure the replace the example value with your own hostname!

#Log directory
HDP_LOG_DIR=c:\hadoop\logs

#Data directory
HDP_DATA_DIR=c:\hadoop\data

#Hosts
NAMENODE_HOST=WIN-4VLBRQK8FA8
SECONDARY_NAMENODE_HOST=WIN-4VLBRQK8FA8
JOBTRACKER_HOST=WIN-4VLBRQK8FA8
HIVE_SERVER_HOST=WIN-4VLBRQK8FA8
OOZIE_SERVER_HOST=WIN-4VLBRQK8FA8
TEMPLETON_HOST=WIN-4VLBRQK8FA8
SLAVE_HOSTS=WIN-4VLBRQK8FA8

#Database host
DB_FLAVOR=derby
DB_HOSTNAME=WIN-4VLBRQK8FA8

#Hive properties
HIVE_DB_NAME=hive
HIVE_DB_USERNAME=hive
HIVE_DB_PASSWORD=hive

#Oozie properties
OOZIE_DB_NAME=oozie
OOZIE_DB_USERNAME=oozie
OOZIE_DB_PASSWORD=oozie

And finally, install HDP for Windows:

msiexec.exe /i "hdp-1.1.0-160.winpkg.msi" /lv install.log \
HDP_LAYOUT=c:\Users\Administrator\Downloads\clusterproperties.txt HDP_DIR=c:\HDP DESTROY_DATA="yes"

This will bring up an MSI install window. When it is done, to verify your installation, check the HDP_DIR it was installed to:

dir c:\HDP

You should see files, such as ‘start_local_hdp_services.cmd’. Run this file:

.\start_local_hdp_services.cmd

With services up, you’re in good shape to run the SmokeTests.

Run-SmokeTests.cmd

Which will fire off a mapreduce job right away. Congratulations, you’re Hadooping on Windows!

mapreduce

If you’d like to learn more about Hadoop, check out the Hortonworks Sandbox, a fully capable virtual machine for you to learn Hadoop with.

 

转自:http://hortonworks.com/blog/installing-hadoop-on-windows/

分享到:
评论

相关推荐

    Kubernetes Microservices with Docker(Apress,2016)

    How to install Kubernetes on a single node How to install Kubernetes on a multi-node cluster How to set environment variables How to create a multi-container pods using Docker How to use volumes How ...

    Pro Docker: Learn how to use Containers as a Service for development and

    In this fast-paced book on the Docker open standards platform for developing, packaging and running portable distributed applications, Deepak Vorhadiscusses how to build, ship and run applications on ...

    Pro Docker(Apress,2015)

    discusses how to build, ship and run applications on any platform such as a PC, the cloud, data center or a virtual machine. He describes how to install and create Docker images. and the advantages ...

    Apress.Pro.Docker

    He describes how to install and create Docker images. and the advantages off Docker containers.The remainder of the book is devoted to discussing using Docker with important software solutions. He ...

    Packt.Learning.Apache.Kafka.2nd.Edition

    how to integrate Kafka with Camus, Apache Camel, Amazon Cloud, and so on.Chapter 6, Kafka Integrations, provides a short introduction to both Storm and Hadoop and discusses how Kafka integration works...

    Practical.Hive.A.Guide.to.Hadoops.Data.Warehouse.System.1484202724

    From deploying Hive on your hardware or virtual machine and setting up its initial configuration to learning how Hive interacts with Hadoop, MapReduce, Tez and other big data technologies, Practical ...

    Apache Flume Distributed Log Collection for Hadoop(PACKT,2ed,2015)

    Understand the Flume architecture, and also how to download and install open source Flume from Apache Follow along a detailed example of transporting weblogs in Near Real Time (NRT) to Kibana/...

    Complete Guide to Open Source Big Data Stack

    See how Brooklyn can be used to install Mule ESB on a cluster and Cassandra in the cloud Install and use DCOS for big data processingUse Apache Spark for big data stack data processing Who This Book...

    Large Scale Machine Learning with Spark

    First, you will learn how to install Spark with all new features from the latest Spark 2.0 release. Moving on, you’ll explore important concepts such as advanced feature engineering with RDD and ...

    Mastering.Geospatial.Analysis.with.Python

    Chapter 1, Package Installation and Management, explains how to install and manage the code libraries used in the book. Chapter 2, Introduction to Geospatial Code Libraries, covers the major code ...

    Ubuntu Hadoop 0.20.2 安装配置说明

    # and how to activate them. For more information, see interfaces(5). # The loopback network interface auto lo iface lo inet loopback # The primary network interface auto eth0 iface eth0 inet static ...

    Practical Hive(Apress,2016)

    From deploying Hive on your hardware or virtual machine and setting up its initial configuration to learning how Hive interacts with Hadoop, MapReduce, Tez and other big data technologies,Practical ...

    large scale machine learning with spark

    You will learn how to install Spark with all new features as in the latest version Spark 2. You will also get to grips with Spark MLlib and Spark ML and its implementation for machine learning ...

    Practical OneOps

    Learn how to install OneOps Configure OneOps, including customizing your organizations, teams and clouds Work through practical deployment scenarios Understand OneOps architecture and individual ...

Global site tag (gtag.js) - Google Analytics