`
kanpiaoxue
  • 浏览: 1781344 次
  • 性别: Icon_minigender_1
  • 来自: 北京
文章分类
社区版块
存档分类
最新评论

Hadoop “Unable to load native-hadoop library for your platform” warning

 
阅读更多

Hadoop “Unable to load native-hadoop library for your platform” warning

 

Hadoop Installation on Mac OS X:

https://isaacchanghau.github.io/post/install_hadoop_mac/

 

Compile Apache Hadoop on Linux (fix warning: Unable to load native-hadoop library)

http://www.ercoppa.org/posts/how-to-compile-apache-hadoop-on-ubuntu-linux.html

 

 

Question:

I'm currently configuring hadoop on a server running CentOs. When I run start-dfs.sh or stop-dfs.sh, I get the following error:

WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable

I'm running Hadoop 2.2.0.

 

Answer:

I assume you're running Hadoop on 64bit CentOS. The reason you saw that warning is the native Hadoop library $HADOOP_HOME/lib/native/libhadoop.so.1.0.0 was actually compiled on 32 bit.

Anyway, it's just a warning, and won't impact Hadoop's functionalities.

Here is the way if you do want to eliminate this warning, download the source code of Hadoop and recompile libhadoop.so.1.0.0 on 64bit system, then replace the 32bit one.

Steps on how to recompile source code are included here for Ubuntu:

http://www.ercoppa.org/Linux-Compile-Hadoop-220-fix-Unable-to-load-native-hadoop-library.htm

Good luck.

 

 

 

 

 

 

 

 

 

 

 

 

Compile Apache Hadoop on Linux (fix warning: Unable to load native-hadoop library)

This tutorial explains how to compile Apache Hadoop 2.4.0 under Ubuntu Linux 13.10 amd64. You may want to compile Hadoop in order to fix the issue:

WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
  • Install a JDK: Oracle JDK (suggested) or package openjdk-7-jdk
  • Install mavenlibssl-devbuild-essentialpkgconf, and cmake.
  • Install the library protobuf:

    • If you are running Ubuntu 13.10 or earlier then install locally protobuf-2.5.0 and insert it in the PATH:

      cd protobuf-2.5.0/
      ./configure --prefix=`pwd`/inst/bin && make && make install
      export PATH=`pwd`/inst/bin:$PATH
    • Otherwise (Ubuntu 14.04 or newer):

      sudo apt-get install libprotobuf8 protobuf-compiler
  • Download Hadoop sources.
  • Compile Apache Hadoop:

    tar xvf hadoop-2.4.0-src.tar.gz
    cd hadoop-2.4.0-src
    mvn package -Pdist,native -DskipTests -Dtar

    Check hadoop-dist/target/hadoop-2.4.0.tar.gz (e.g., use this as your hadoop binary) or hadoop-dist/target/hadoop-2.4.0. If you have already installed a 32bit Hadoop, then you need only to replace the native libs in $HADOOP/lib/ with the new native libs (hadoop-dist/target/hadoop-2.4.0/lib) and remove (if applicable) from $HADOOP/etc/hadoop-env.sh:

    export HADOOP_COMMON_LIB_NATIVE_DIR="~/hadoop/lib/"
    export HADOOP_OPTS="$HADOOP_OPTS -Djava.library.path=~/hadoop/lib/"
  • You can delete Hadoop (and Protobuf sources if Ubuntu 13.10 or earlier).
分享到:
评论

相关推荐

    Unable to load native-hadoop library for your platform...

    Unable to load native-hadoop library for your platform... using builtin-java classes where applicable 1. 下载文件native.rar 并解压 2. 用SecureFX 上传, 替换native目录下文件,选择二进制文件上传。 3. ...

    Hadoop 启动WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform… using

    搭建hadoop 环境时遇到启动异常告警问题 “WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform… using builtin-java classes where applicable” 上来不多说,百度收集些相关...

    hadoop-native-64-2.7.0.tar

    解决:Unable to load native-hadoop library for your platform 问题 原因: Apache提供的hadoop本地库是32位的,而在64位的服务器上就会有问题,因此需要自己编译64位的版本。 将准备好的64位的lib包解压到已经...

    hadoop-native-64-2.6.0.zip

    Hadoop使用过程报错:WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform...解决 下载过后把tar包解压在hdoop的lib/native下即可

    win10-hadoop3.0.0或其他版本 Unable to load native-hadoop library 缺失文件

    hadoop 3.0.0 或者其他版本 报错Unable to load native-hadoop library时,放到c:/system32就可以

    hadoop-native-64-2.8.1.tar.7z

    用这个64位替换了官方的32位的native Unable to load native-hadoop library for your platform... using builtin-java classes where applicable

    hadoop2.8.1 native for macOS10

    WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable Found 2 items 解决方法是下载hadoop源代码重新编译, 但中间需要下载安装...

    hadoop-native-64-2.5.0.tar

    解决:Unable to load native-hadoop library for your platform 问题 原因: Apache提供的hadoop本地库是32位的,而在64位的服务器上就会有问题,因此需要自己编译64位的版本。 将准备好的64位的lib包解压到已经...

    hadoop-native-64-2.6.0.tar

    解决:Unable to load native-hadoop library for your platform 问题 原因: Apache提供的hadoop本地库是32位的,而在64位的服务器上就会有问题,因此需要自己编译64位的版本。 将准备好的64位的lib包解压到已经...

    hadoop-native-64-2.4.1

    解决:Unable to load native-hadoop library for your platform 问题 原因: Apache提供的hadoop本地库是32位的,而在64位的服务器上就会有问题,因此需要自己编译64位的版本。 将准备好的64位的lib包解压到已经...

    hadoop-native-64-2.5.2.tar

    解决:Unable to load native-hadoop library for your platform 问题 原因: Apache提供的hadoop本地库是32位的,而在64位的服务器上就会有问题,因此需要自己编译64位的版本。 将准备好的64位的lib包解压到已经...

    haddop-2.6.4 native 64位

    为了解决执行hadoop 命令时总有警告: WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable

    hadoop-native-64-2.4.0.tar

    hadoop-native-64-2.4.0.tar 解决运行hadoop 警告:WARN util.NativeCodeLoader: Unable to load native-hadoop

    hadoop 2.8.0 macOS native library

    安装hadoop启动之后总有警告:Unable to load native-hadoop library for your platform... using builtin-java classes where applicable 原因: Apache提供的hadoop本地库是32位的,而在64位的服务器上就会有问题...

    hadoop 2.7.2 64位 native包

    WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable 官网上源码hadoop2.7.2,64位机上编译的,CentOS6.6 64位亲测。

    hadoop lib/native

    在安装hadoop时如果遇到WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform这个错误,请下载该文件放到hadoop目录下,Mac中放到libexec目录下。

    hadoop-eclipse-plugin-2.10.0.jar

    - `hadoop2x-eclipse-plugin-master/ivy/library.properties` - `hadoop2x-eclipse-plugin-master/src/contrib/eclipse-plugin/build.xml` 开源源地址: https://github.com/winghc/hadoop2x-eclipse-plugin

    hadoop-2.6.0编译好的64bit的native库

    14/01/27 10:52:34 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable Starting namenodes on [Java HotSpot(TM) 64-Bit ...

Global site tag (gtag.js) - Google Analytics