leibnitz
- 浏览: 283612 次
- 性别:
- 来自: 广州
-
最新评论
-
jpsb：
...
为什么需要分布式？ -
leibnitz：
hi guy, this is used as develo ...
compile hadoop-2.5.x on OS X(macbook) -
string2020：
撸主真土豪,在苹果里面玩大数据.
compile hadoop-2.5.x on OS X(macbook) -
youngliu_liu：
怎样运行这个脚本啊？？大牛，我刚进入搜索引擎行业，希望你能不吝 ...
nutch 数据增量更新 -
leibnitz：
also, there is a similar bug ...
２。hbase CRUD--Lease in hbase
文章列表
清华大学数学科学系报考问答
数学专业参考书整理推荐
java nio tutorial
- 博客分类：
- java
a simple,easily understandable tutorial found here
Java NIO Tutorial written by jakob
yes,i have studied zookeeper(3.4.3) before,but that is only a simple 'usage' ,not involved any source codes to dig into it's details.now i will try to finish remaining hardly.
so this are some tasks to dig for it by these weeks :
1.summary
2.leader election
3.add a peer dynamically
4.a ...
we know,hadoop will show only abstract result info about mapreduce(represent in http://xxx:19888/clsuter by default),so it's unconvenient to track the number of mappers,where mappers run,how to track the exception logs etc.note ,this is the default behavior by hadoop(maybe decrease the resouces ...
1. submitting a MR job
2.flow from nodemanager to resourcemanager:
so from this figure,we know that the yarn use WritableRPCEnginge.java as rpc engine by default,but here of course,hadoop use ProtobufRpcEngine.java instead ,and the serialization/deserialization of parameters ar ...
this feature is a considerable performance improvement,as per the idea 'moves the computation to data side',this is a further usage of it.
Outline
1.what is
2.how to
3.setup to run
4.one more thinking
1.what is
about ten years ago,i usually heard some ad words,maybe belong o ...
I dont want to restruct wheels of open sources,in contrast, just wonder the implied features and use cases as possible.so i will write somethings to summary or memo.
Agenda
1.what is
2.how to
3.hadoop snapshot vs hbase snapshot
4.demos to use snapshot
1.what is
a long time ago ...
note,this article is focus on hadoop-2.5.1,it maybe a little different from hadoop-0.23.x.
Agenda:
I:hdfs federation abstract
II:implement
III:other distributed namespaces scalability
-------------------
I:hdfs federation abstract
in hadoop-1.x or before(except 0.23.x),there is a coup ...
below are the outlines of this article ,using by hadoop-2.5.1 :
1.abstraction
1.1 what is HDFS HA
1.2 how to
1.3 other HAs
2.installation
2.1 manual failover
2.2 auto failover
3.conclusion
1.abstraction
1.1 what is HDFS HA
in hadoop distributed file system ,there are some p ...
this article is the another part follow to hadoop 2.x-HDFS HA --Part I: abstraction ,and here will talk about these topics:
2.installation HA
2.1 manual failover
2.2 auto failover
3.conclusion
2.installation HA
2.1 manual failover
for this mode,the cluster can be failovered ...
as of hadoop-2.5.1,there are some important features of hadoop 2.x for me to learn these days
new key features
abstract
NN HA (high availability)
standby meta node,failure over anytime,maybe same as
hbase-backup master
HDFS Federation
union all nameapes from different rights
...
I. installation mode
same as hadoop 1.x ,there are several mode to install hadoop:
1.standalone
just run it on one machine,includes running of mapreduce.
2.pseudo
setup it with hdfs mode,and this case contains two types:
a.run hdfs only
in this case,mapreds also run in local mod ...
same as compile hbase ,it 's necessary for u to compile your own dist of hadoop to get best performance specified to your hardwares.
bellow are some steps needed by this theme:
1.install some tools
a.cmake 2.6+
download and set the path variable
this is used only if u want ...
1.the match relationships between hadoop and hbase nowaday
that means if you are now using hadoop1.0.x and hbase-0.94.x,and assume u want to upgrade to hadoop-2.5.x and hbase-1.0.x,the steps should be followed:
2.upgrade
2.1 upgrade paths
as described above,there are two lines to ...
u know,when u shutdowning a hbase cluster,sometimes it costs very long time to complete.and now how to fiind what regionservers are kept on alive is simple and directly by following these steps:
1.run shutdown shell stop-hbase.sh
2.issue zk-client zkCli.sh
3.run command 'ls /hbase/rs'
th ...