工具说明
YCSB是雅虎开放的一个压测标准,提供了很多组件的压测支持
github:https://github.com/brianfrankcooper/YCSB
编译工程
zookeeper压测只是ycsb其中一个模块,工程采用java代码开发,运行至少需要jdk8
本地配置好jdk和maven环境后,编译zookeeper模块命令
1. 打包命令
mvn -Psource-run -pl site.ycsb:zookeeper-binding -am clean package -DskipTests
2. 编译命令
README.MD文件中给出的编译命令:
mvn -pl site.ycsb:zookeeper-binding -am clean package -DskipTests
说明:
zookeeper模块readme.md文件中关于编译给出的是上面的命令2,但是如果要执行,实际脚本中还是会执行上面的命令1,因此建议最好是直接执行打包部署命令,两者的区别是执行命令1,会同时在core和zookeeper模块的target/dependency下拷贝依赖包,命令2只会在zookeeper模块下拷贝依赖包
部署运行
在编译中,zookeeper模块执行依赖的包已经分别拷贝到core和zookeeper模块的target/dependency的路径下,因此这里可以直接对当前ycsb工程压缩打包并部署到服务器上,因为java是跨平台的,因此只要服务器上有jdk或jre环境,就可以执行测试
默认上传到服务器后,启动脚本会没有执行权限,设置执行权限
chmod +x bin/ycsb.sh
脚本启动后,是以java -cp的形式来运行程序,从classpath中寻找主类运行,这里的classpath中的包,主要就是上面core和zookeeper模块target/dependency路径下的包
运行示例:
/usr/bin/java -classpath
/root/benchmark/ycsb-zoo/conf:
/root/benchmark/ycsb-zoo/core/target/core-0.18.0-SNAPSHOT.jar:
/root/benchmark/ycsb-zoo/core/target/dependency/HdrHistogram-2.1.4.jar:
/root/benchmark/ycsb-zoo/core/target/dependency/htrace-core4-4.1.0-incubating.jar:
/root/benchmark/ycsb-zoo/core/target/dependency/jackson-core-asl-1.9.4.jar:
/root/benchmark/ycsb-zoo/core/target/dependency/jackson-mapper-asl-1.9.4.jar:
/root/benchmark/ycsb-zoo/zookeeper/target/zookeeper-binding-0.18.0-SNAPSHOT.jar:
/root/benchmark/ycsb-zoo/zookeeper/target/dependency/audience-annotations-0.5.0.jar:
/root/benchmark/ycsb-zoo/zookeeper/target/dependency/json-simple-1.1.1.jar:
/root/benchmark/ycsb-zoo/zookeeper/target/dependency/log4j-1.2.17.jar:
/root/benchmark/ycsb-zoo/zookeeper/target/dependency/netty-buffer-4.1.48.Final.jar:
/root/benchmark/ycsb-zoo/zookeeper/target/dependency/netty-codec-4.1.48.Final.jar:
/root/benchmark/ycsb-zoo/zookeeper/target/dependency/netty-common-4.1.48.Final.jar:
/root/benchmark/ycsb-zoo/zookeeper/target/dependency/netty-handler-4.1.48.Final.jar:
/root/benchmark/ycsb-zoo/zookeeper/target/dependency/netty-resolver-4.1.48.Final.jar:
/root/benchmark/ycsb-zoo/zookeeper/target/dependency/netty-transport-4.1.48.Final.jar:
/root/benchmark/ycsb-zoo/zookeeper/target/dependency/netty-transport-native-epoll-4.1.48.Final.jar:
/root/benchmark/ycsb-zoo/zookeeper/target/dependency/netty-transport-native-unix-common-4.1.48.Final.jar:
/root/benchmark/ycsb-zoo/zookeeper/target/dependency/slf4j-api-1.7.21.jar:
/root/benchmark/ycsb-zoo/zookeeper/target/dependency/slf4j-log4j12-1.7.25.jar:
/root/benchmark/ycsb-zoo/zookeeper/target/dependency/zookeeper-3.5.8.jar:
/root/benchmark/ycsb-zoo/zookeeper/target/dependency/zookeeper-jute-3.5.8.jar
site.ycsb.Client
-t -db site.ycsb.db.zookeeper.ZKClient -s -P workloads/workloadb -p zookeeper.connectString=127.0.0.1:2181/benchmark
Command line: -t -db site.ycsb.db.zookeeper.ZKClient -s -P workloads/workloadb -p zookeeper.connectString=127.0.0.1:2181/benchmark
其中core模块下5个包,zookeeper下16个包
压测参数
示例:
加载
./bin/ycsb.sh load zookeeper -s -P workloads/workloadb -p zookeeper.connectString=127.0.0.1:2181/benchmark -p recordcount=10000
测试
./bin/ycsb run zookeeper -s -P workloads/workloadb -p zookeeper.connectString=127.0.0.1:2181/benchmark
load
插入加载数据
run
测试数据
-jvm-args
设置jvm参数
-jvm-args="-Djute.maxbuffer=4194304"
-s
打印运行中的状态
-threads
指定并发线程数
-P
加载指定文件,进行测试,文件中指定了配置参数
-p
指定键值对配置参数
指定连接地址
zookeeper.connectString=127.0.0.1:2181/benchmark
zookeeper.connectString=192.168.10.43:2181,192.168.10.45:2181,192.168.10.27:2181/benchmark
指定测试数据字段长度
fieldlength=1000
指定测试数据字段数量
fieldcount=20
指定会话超时时间
zookeeper.sessionTimeout=30000
指定压测输出结果统计区间
hdrhistogram.percentiles=10,25,50,75,90,95,99,99.9
指定记录数
recordcount=1000
指定数据处理数量
operationcount=1000
指定读取操作数
readproportion=0.95
指定更新操作数
updateproportion=0.05
参考
<LI><b>fieldcount</b>: the number of fields in a record (default: 10)
<LI><b>fieldlength</b>: the size of each field (default: 100)
<LI><b>readallfields</b>: should reads read all fields (true) or just one (false) (default: true)
<LI><b>readproportion</b>: what proportion of operations should be reads (default: 0.95)
<LI><b>updateproportion</b>: what proportion of operations should be updates (default: 0.05)
<LI><b>insertproportion</b>: what proportion of operations should be inserts (default: 0)
<LI><b>scanproportion</b>: what proportion of operations should be scans (default: 0)
<LI><b>readmodifywriteproportion</b>: what proportion of operations should be read a record, modify it, write it back (default: 0)
<LI><b>requestdistribution</b>: what distribution should be used to select the records to operate on - uniform, zipfian or latest (default: uniform)
<LI><b>maxscanlength</b>: for scans, what is the maximum number of records to scan (default: 1000)
<LI><b>scanlengthdistribution</b>: for scans, what distribution should be used to choose the number of records to scan, for each scan, between 1 and maxscanlength (default: uniform)
<LI><b>insertorder</b>: should records be inserted in order by key ("ordered"), or in hashed order ("hashed") (default: hashed)
<LI><b>fieldnameprefix</b>: string prefix for the field name (default: “field”)