`
standalone
  • 浏览: 613028 次
  • 性别: Icon_minigender_1
  • 来自: 上海
社区版块
存档分类
最新评论

Using the libjars option with Hadoop

 
阅读更多
As I have said in my last post, I was developing a hbase based mapreduce application. But one damn thing is the hadoop cluster managed by our system admin has no hbase jars in its classpath...

So I have to figure out how to provide jars needed by hbase to hadoop. I know "hadoop jar" command has an option "-libjars" which allow you to provide third party jars needed by your mapper or reducer. I did tried it. But it still says class HBaseConfiguration cannot be found. I tried many times without a success.

I got stuck at this...

Then a long weekend.

Today is monday. I continue work on this. I should mention that I have read the articles like these:

http://blog.cloudera.com/blog/2011/01/how-to-include-third-party-libraries-in-your-map-reduce-job/

http://grepalex.com/2013/02/25/hadoop-libjars/

But still failed. Finally I add the jars to HADOOP_CLASSPATH and set HADOOP_USER_CLASSPATH_FIRST true, then rerun the job, no errors any more!

分享到:
评论

相关推荐

    Scaling Big Data with Hadoop and Solr

    Starting with the basics of Apache Hadoop and Solr, this book then dives into advanced topics of optimizing search with some interesting real-world use cases and sample Java code.

    Data Analytics with Hadoop: An Introduction for Data Scientists

    operations, or software development usually associated with distributed computing, you’ll focus on particular analyses you can build, the data warehousing techniques that Hadoop provides, and higher...

    Deep Learning with Hadoop

    Starting with understanding what deep learning is and what the various models associated with deep learning are, this book will then show you how to set up the Hadoop environment for deep learning....

    Data Analytics with Hadoop(O'Reilly,2016)

    operations, or software development usually associated with distributed computing, you’ll focus on particular analyses you can build, the data warehousing techniques that Hadoop provides, and higher...

    Data Algorithms Recipes for Scaling Up with Hadoop and Spark epub

    Data Algorithms Recipes for Scaling Up with Hadoop and Spark 英文epub 本资源转载自网络,如有侵权,请联系上传者或csdn删除 本资源转载自网络,如有侵权,请联系上传者或csdn删除

    Modern Big Data Processing with Hadoop-Packt Publishing(2018)

    Apache Hadoop, along with a host of other big data tools, empowers you to build such solutions with relative ease. This book lists some unique ideas and techniques that enable you to conquer ...

    Big Data Analytics with Hadoop 3

    Explore big data concepts, platforms, analytics, and their applications using the power of Hadoop 3 Apache Hadoop is the most popular platform for big data processing, and can be combined with a host ...

    Data Analytics with Hadoop

    《Data Analytics with Hadoop》是一本专注于大数据分析与处理的权威教材,主要涵盖了Hadoop在数据挖掘、分析领域的核心技术和应用。Hadoop是Apache软件基金会的一个开源项目,它设计用于处理和存储海量数据,是大...

    Deep Learning with Hadoop epub

    Deep Learning with Hadoop 英文epub 本资源转载自网络,如有侵权,请联系上传者或csdn删除 查看此书详细信息请在美国亚马逊官网搜索此书

    Deep learning with Hadoop : build, implement and scale distributed d l models

    deep neural networks with Hadoop for optimal performance. Starting with understanding what deep learning is, and what the various models associated with deep neural networks are, this book will then ...

    Hadoop from the beginning: The basics

    The author helps you familiarize yourself with the various commands that you can use to perform various tasks within the Hadoop system. The author also helps you know how to write MapReduce programs ...

    Modern Big Data Processing with Hadoop epub

    Modern Big Data Processing with Hadoop 英文epub 本资源转载自网络,如有侵权,请联系上传者或csdn删除 查看此书详细信息请在美国亚马逊官网搜索此书

    Deep Learning with Hadoop 无水印原版pdf

    Deep Learning with Hadoop 英文无水印原版pdf pdf所有页面使用FoxitReader、PDF-XChangeViewer、SumatraPDF和Firefox测试都可以打开 本资源转载自网络,如有侵权,请联系上传者或csdn删除 查看此书详细信息请...

Global site tag (gtag.js) - Google Analytics