Arch-03-19- NoSQL实践

cnmqw

浏览: 224089 次
性别:
来自: 深圳

最近访客更多访客>>

chuyule

wolfwolfgod

lijiejava

lf78204490

博主相关

博客

微博

相册

留言

关于我

文章分类

社区版块

存档分类

博客分类：

架构师-03-实施

虽然对新技术甚为谨慎，但目前大家所做的伪 sns 的 nosql 似乎也让一帮同事兴奋不已，没办法，试试吧。

不讨论nosql 的长短，直接实施吧，关键在于 nosql 表的设计和与 postgrsql 间数据的关系组织。

一、用 YCSB 测试候选的数据库

（1）候选：hbase,voldemort,mongodb

（2）ycsb 环境：centos5.4,12G,4x4核服务器

（3）下载安装 ycsb 略过

（4）Hbase-0.9

. 插入数据 100w

java -cp build/ycsb.jar:db/hbase/lib/* com.yahoo.ycsb.Client -load -db com.yahoo.ycsb.db.HBaseClient -P workloads/workloada -p columnfamily=family -p recordcount=10000000 -s > load.dat

 10 sec: 73825 operations; 7380.29 current ops/sec; [INSERT AverageLatency(ms)=0.11]
 20 sec: 127145 operations; 5329.87 current ops/sec; [INSERT AverageLatency(ms)=0.16]
 30 sec: 217374 operations; 9020.19 current ops/sec; [INSERT AverageLatency(ms)=0.09]
...

 平均 insert 8k/sec

java -cp build/ycsb.jar:db/hbase/lib/* com.yahoo.ycsb.Client -t -db com.yahoo.ycsb.db.HBaseClient -P workloads/workloada -p columnfamily=family -p operationcount=1000000 -s -threads 10 -target 100 > transactions.dat

（5）Voldemort-0.9

初始化插入 500w，0.7k/s

YCSB Client 0.1
Command line: -load -db com.yahoo.ycsb.db.VoldemortClient -p bootstrap_urls=tcp://localhost:6666 -P workloads/workloada -s -threads 10
[OVERALL], RunTime(ms), 655195.0
[OVERALL], Throughput(ops/sec), 7631.315867795084
[INSERT], Operations, 5000000
[INSERT], AverageLatency(ms), 1.2568772
[INSERT], MinLatency(ms), 0
[INSERT], MaxLatency(ms), 2289
[INSERT], 95thPercentileLatency(ms), 1
[INSERT], 99thPercentileLatency(ms), 3
[INSERT], Return=0, 5000000

写/读各50w 次，写 0.83k/s,R1.2k/s

YCSB Client 0.1
Command line: -t -db com.yahoo.ycsb.db.VoldemortClient -p bootstrap_urls=tcp://localhost:6666 -P workloads/workloada -s
[OVERALL], RunTime(ms), 1026399.0
[OVERALL], Throughput(ops/sec), 974.2799827357587
[UPDATE], Operations, 501020
[UPDATE], AverageLatency(ms), 1.2046704722366373
[UPDATE], MinLatency(ms), 0
[UPDATE], MaxLatency(ms), 1887
[UPDATE], 95thPercentileLatency(ms), 5
[UPDATE], 99thPercentileLatency(ms), 8
[UPDATE], Return=0, 501020

....


[READ], Operations, 498980
[READ], AverageLatency(ms), 0.8289450478977113
[READ], MinLatency(ms), 0
[READ], MaxLatency(ms), 359
[READ], 95thPercentileLatency(ms), 5
[READ], 99thPercentileLatency(ms), 7
[READ], Return=0, 498980

（6）mongodb

初始化插入 500w，0.1k/s

YCSB Client 0.1
Command line: -load -db com.yahoo.ycsb.db.MongoDbClient -p mongodb.database=mytestdbt100 -p mongodb.url=mongodb://localhost:27017 -P workloads/workloada -s -threads 100
new database url = localhost:27017/mytestdbt100
new database url = localhost:27017/mytestdbt100
...100行

mongo connection created with localhost:27017/mytestdbt100
mongo connection created with localhost:27017/mytestdbt100
...100行

[OVERALL], RunTime(ms), 4664799.0
[OVERALL], Throughput(ops/sec), 1071.857544129983
[INSERT], Operations, 5000000
[INSERT], AverageLatency(ms), 93.0971558
[INSERT], MinLatency(ms), 0
[INSERT], MaxLatency(ms), 34143
[INSERT], 95thPercentileLatency(ms), 220
[INSERT], Return=0, 5000000

写/读各50w 次，写 0.13k/s,R3.7k/s

YCSB Client 0.1
Command line: -t -db com.yahoo.ycsb.db.MongoDbClient -p mongodb.database=mytestdb100 -p mongodb.url=mongodb://localhost:27017 -P workloads/workloada -s -threads 100
new database url = localhost:27017/mytestdb100
new database url = localhost:27017/mytestdb100
...100

mongo connection created with localhost:27017/mytestdb100
mongo connection created with localhost:27017/mytestdb100
...100

[OVERALL], RunTime(ms), 40395.0
[OVERALL], Throughput(ops/sec), 24755.539051862856
[UPDATE], Operations, 500052
[UPDATE], AverageLatency(ms), 7.622983209746186
[UPDATE], MinLatency(ms), 0
[UPDATE], MaxLatency(ms), 1344
[UPDATE], 95thPercentileLatency(ms), 18
[UPDATE], 99thPercentileLatency(ms), 27
[UPDATE], Return=1, 500052
...

READ], Operations, 499948
[READ], AverageLatency(ms), 0.27114819941273893
[READ], MinLatency(ms), 0
[READ], MaxLatency(ms), 1308
[READ], 95thPercentileLatency(ms), 1
[READ], 99thPercentileLatency(ms), 1
[READ], Return=1, 499948
...

（7）MongoDB Java 框架选择