searchusermenu
  • 发布文章
  • 消息中心
点赞
收藏
评论
分享
原创

ycsb压测之-zookeeper

2023-07-13 07:02:25
119
0

工具说明

YCSB是雅虎开放的一个压测标准,提供了很多组件的压测支持

github:https://github.com/brianfrankcooper/YCSB

编译工程

zookeeper压测只是ycsb其中一个模块,工程采用java代码开发,运行至少需要jdk8

本地配置好jdk和maven环境后,编译zookeeper模块命令

1. 打包命令
mvn -Psource-run -pl site.ycsb:zookeeper-binding -am clean package -DskipTests

2. 编译命令
README.MD文件中给出的编译命令:
mvn -pl site.ycsb:zookeeper-binding -am clean package -DskipTests


说明:

zookeeper模块readme.md文件中关于编译给出的是上面的命令2,但是如果要执行,实际脚本中还是会执行上面的命令1,因此建议最好是直接执行打包部署命令,两者的区别是执行命令1,会同时在core和zookeeper模块的target/dependency下拷贝依赖包,命令2只会在zookeeper模块下拷贝依赖包

部署运行

在编译中,zookeeper模块执行依赖的包已经分别拷贝到core和zookeeper模块的target/dependency的路径下,因此这里可以直接对当前ycsb工程压缩打包并部署到服务器上,因为java是跨平台的,因此只要服务器上有jdk或jre环境,就可以执行测试

默认上传到服务器后,启动脚本会没有执行权限,设置执行权限

chmod +x bin/ycsb.sh

脚本启动后,是以java -cp的形式来运行程序,从classpath中寻找主类运行,这里的classpath中的包,主要就是上面core和zookeeper模块target/dependency路径下的包

运行示例:

/usr/bin/java  -classpath 
/root/benchmark/ycsb-zoo/conf:
/root/benchmark/ycsb-zoo/core/target/core-0.18.0-SNAPSHOT.jar:
/root/benchmark/ycsb-zoo/core/target/dependency/HdrHistogram-2.1.4.jar:
/root/benchmark/ycsb-zoo/core/target/dependency/htrace-core4-4.1.0-incubating.jar:
/root/benchmark/ycsb-zoo/core/target/dependency/jackson-core-asl-1.9.4.jar:
/root/benchmark/ycsb-zoo/core/target/dependency/jackson-mapper-asl-1.9.4.jar:
/root/benchmark/ycsb-zoo/zookeeper/target/zookeeper-binding-0.18.0-SNAPSHOT.jar:
/root/benchmark/ycsb-zoo/zookeeper/target/dependency/audience-annotations-0.5.0.jar:
/root/benchmark/ycsb-zoo/zookeeper/target/dependency/json-simple-1.1.1.jar:
/root/benchmark/ycsb-zoo/zookeeper/target/dependency/log4j-1.2.17.jar:
/root/benchmark/ycsb-zoo/zookeeper/target/dependency/netty-buffer-4.1.48.Final.jar:
/root/benchmark/ycsb-zoo/zookeeper/target/dependency/netty-codec-4.1.48.Final.jar:
/root/benchmark/ycsb-zoo/zookeeper/target/dependency/netty-common-4.1.48.Final.jar:
/root/benchmark/ycsb-zoo/zookeeper/target/dependency/netty-handler-4.1.48.Final.jar:
/root/benchmark/ycsb-zoo/zookeeper/target/dependency/netty-resolver-4.1.48.Final.jar:
/root/benchmark/ycsb-zoo/zookeeper/target/dependency/netty-transport-4.1.48.Final.jar:
/root/benchmark/ycsb-zoo/zookeeper/target/dependency/netty-transport-native-epoll-4.1.48.Final.jar:
/root/benchmark/ycsb-zoo/zookeeper/target/dependency/netty-transport-native-unix-common-4.1.48.Final.jar:
/root/benchmark/ycsb-zoo/zookeeper/target/dependency/slf4j-api-1.7.21.jar:
/root/benchmark/ycsb-zoo/zookeeper/target/dependency/slf4j-log4j12-1.7.25.jar:
/root/benchmark/ycsb-zoo/zookeeper/target/dependency/zookeeper-3.5.8.jar:
/root/benchmark/ycsb-zoo/zookeeper/target/dependency/zookeeper-jute-3.5.8.jar 
site.ycsb.Client 
-t -db site.ycsb.db.zookeeper.ZKClient -s -P workloads/workloadb -p zookeeper.connectString=127.0.0.1:2181/benchmark

Command line: -t -db site.ycsb.db.zookeeper.ZKClient -s -P workloads/workloadb -p zookeeper.connectString=127.0.0.1:2181/benchmark


其中core模块下5个包,zookeeper下16个包

压测参数

示例:
加载
  ./bin/ycsb.sh load zookeeper -s -P workloads/workloadb -p zookeeper.connectString=127.0.0.1:2181/benchmark -p recordcount=10000
测试
  ./bin/ycsb run zookeeper -s -P workloads/workloadb -p zookeeper.connectString=127.0.0.1:2181/benchmark

load

  插入加载数据

run

  测试数据

-jvm-args

  设置jvm参数

  -jvm-args="-Djute.maxbuffer=4194304"

-s 

  打印运行中的状态

-threads

  指定并发线程数

-P

  加载指定文件,进行测试,文件中指定了配置参数

-p

  指定键值对配置参数

  指定连接地址

    zookeeper.connectString=127.0.0.1:2181/benchmark

    zookeeper.connectString=192.168.10.43:2181,192.168.10.45:2181,192.168.10.27:2181/benchmark

  指定测试数据字段长度

    fieldlength=1000

  指定测试数据字段数量

    fieldcount=20

  指定会话超时时间

    zookeeper.sessionTimeout=30000

  指定压测输出结果统计区间

    hdrhistogram.percentiles=10,25,50,75,90,95,99,99.9

  指定记录数

    recordcount=1000

  指定数据处理数量

    operationcount=1000

  指定读取操作数

    readproportion=0.95

  指定更新操作数

    updateproportion=0.05

  参考
<LI><b>fieldcount</b>: the number of fields in a record (default: 10) 
<LI><b>fieldlength</b>: the size of each field (default: 100) 
<LI><b>readallfields</b>: should reads read all fields (true) or just one (false) (default: true) 
<LI><b>readproportion</b>: what proportion of operations should be reads (default: 0.95) 
<LI><b>updateproportion</b>: what proportion of operations should be updates (default: 0.05) 
<LI><b>insertproportion</b>: what proportion of operations should be inserts (default: 0) 
<LI><b>scanproportion</b>: what proportion of operations should be scans (default: 0) 
<LI><b>readmodifywriteproportion</b>: what proportion of operations should be read a record, modify it, write it back (default: 0) 
<LI><b>requestdistribution</b>: what distribution should be used to select the records to operate on - uniform, zipfian or latest (default: uniform) 
<LI><b>maxscanlength</b>: for scans, what is the maximum number of records to scan (default: 1000) 
<LI><b>scanlengthdistribution</b>: for scans, what distribution should be used to choose the number of records to scan, for each scan, between 1 and maxscanlength (default: uniform) 
<LI><b>insertorder</b>: should records be inserted in order by key ("ordered"), or in hashed order ("hashed") (default: hashed)
<LI><b>fieldnameprefix</b>: string prefix for the field name (default: “field”)
0条评论
0 / 1000
奔驰上树
4文章数
0粉丝数
奔驰上树
4 文章 | 0 粉丝
奔驰上树
4文章数
0粉丝数
奔驰上树
4 文章 | 0 粉丝
原创

ycsb压测之-zookeeper

2023-07-13 07:02:25
119
0

工具说明

YCSB是雅虎开放的一个压测标准,提供了很多组件的压测支持

github:https://github.com/brianfrankcooper/YCSB

编译工程

zookeeper压测只是ycsb其中一个模块,工程采用java代码开发,运行至少需要jdk8

本地配置好jdk和maven环境后,编译zookeeper模块命令

1. 打包命令
mvn -Psource-run -pl site.ycsb:zookeeper-binding -am clean package -DskipTests

2. 编译命令
README.MD文件中给出的编译命令:
mvn -pl site.ycsb:zookeeper-binding -am clean package -DskipTests


说明:

zookeeper模块readme.md文件中关于编译给出的是上面的命令2,但是如果要执行,实际脚本中还是会执行上面的命令1,因此建议最好是直接执行打包部署命令,两者的区别是执行命令1,会同时在core和zookeeper模块的target/dependency下拷贝依赖包,命令2只会在zookeeper模块下拷贝依赖包

部署运行

在编译中,zookeeper模块执行依赖的包已经分别拷贝到core和zookeeper模块的target/dependency的路径下,因此这里可以直接对当前ycsb工程压缩打包并部署到服务器上,因为java是跨平台的,因此只要服务器上有jdk或jre环境,就可以执行测试

默认上传到服务器后,启动脚本会没有执行权限,设置执行权限

chmod +x bin/ycsb.sh

脚本启动后,是以java -cp的形式来运行程序,从classpath中寻找主类运行,这里的classpath中的包,主要就是上面core和zookeeper模块target/dependency路径下的包

运行示例:

/usr/bin/java  -classpath 
/root/benchmark/ycsb-zoo/conf:
/root/benchmark/ycsb-zoo/core/target/core-0.18.0-SNAPSHOT.jar:
/root/benchmark/ycsb-zoo/core/target/dependency/HdrHistogram-2.1.4.jar:
/root/benchmark/ycsb-zoo/core/target/dependency/htrace-core4-4.1.0-incubating.jar:
/root/benchmark/ycsb-zoo/core/target/dependency/jackson-core-asl-1.9.4.jar:
/root/benchmark/ycsb-zoo/core/target/dependency/jackson-mapper-asl-1.9.4.jar:
/root/benchmark/ycsb-zoo/zookeeper/target/zookeeper-binding-0.18.0-SNAPSHOT.jar:
/root/benchmark/ycsb-zoo/zookeeper/target/dependency/audience-annotations-0.5.0.jar:
/root/benchmark/ycsb-zoo/zookeeper/target/dependency/json-simple-1.1.1.jar:
/root/benchmark/ycsb-zoo/zookeeper/target/dependency/log4j-1.2.17.jar:
/root/benchmark/ycsb-zoo/zookeeper/target/dependency/netty-buffer-4.1.48.Final.jar:
/root/benchmark/ycsb-zoo/zookeeper/target/dependency/netty-codec-4.1.48.Final.jar:
/root/benchmark/ycsb-zoo/zookeeper/target/dependency/netty-common-4.1.48.Final.jar:
/root/benchmark/ycsb-zoo/zookeeper/target/dependency/netty-handler-4.1.48.Final.jar:
/root/benchmark/ycsb-zoo/zookeeper/target/dependency/netty-resolver-4.1.48.Final.jar:
/root/benchmark/ycsb-zoo/zookeeper/target/dependency/netty-transport-4.1.48.Final.jar:
/root/benchmark/ycsb-zoo/zookeeper/target/dependency/netty-transport-native-epoll-4.1.48.Final.jar:
/root/benchmark/ycsb-zoo/zookeeper/target/dependency/netty-transport-native-unix-common-4.1.48.Final.jar:
/root/benchmark/ycsb-zoo/zookeeper/target/dependency/slf4j-api-1.7.21.jar:
/root/benchmark/ycsb-zoo/zookeeper/target/dependency/slf4j-log4j12-1.7.25.jar:
/root/benchmark/ycsb-zoo/zookeeper/target/dependency/zookeeper-3.5.8.jar:
/root/benchmark/ycsb-zoo/zookeeper/target/dependency/zookeeper-jute-3.5.8.jar 
site.ycsb.Client 
-t -db site.ycsb.db.zookeeper.ZKClient -s -P workloads/workloadb -p zookeeper.connectString=127.0.0.1:2181/benchmark

Command line: -t -db site.ycsb.db.zookeeper.ZKClient -s -P workloads/workloadb -p zookeeper.connectString=127.0.0.1:2181/benchmark


其中core模块下5个包,zookeeper下16个包

压测参数

示例:
加载
  ./bin/ycsb.sh load zookeeper -s -P workloads/workloadb -p zookeeper.connectString=127.0.0.1:2181/benchmark -p recordcount=10000
测试
  ./bin/ycsb run zookeeper -s -P workloads/workloadb -p zookeeper.connectString=127.0.0.1:2181/benchmark

load

  插入加载数据

run

  测试数据

-jvm-args

  设置jvm参数

  -jvm-args="-Djute.maxbuffer=4194304"

-s 

  打印运行中的状态

-threads

  指定并发线程数

-P

  加载指定文件,进行测试,文件中指定了配置参数

-p

  指定键值对配置参数

  指定连接地址

    zookeeper.connectString=127.0.0.1:2181/benchmark

    zookeeper.connectString=192.168.10.43:2181,192.168.10.45:2181,192.168.10.27:2181/benchmark

  指定测试数据字段长度

    fieldlength=1000

  指定测试数据字段数量

    fieldcount=20

  指定会话超时时间

    zookeeper.sessionTimeout=30000

  指定压测输出结果统计区间

    hdrhistogram.percentiles=10,25,50,75,90,95,99,99.9

  指定记录数

    recordcount=1000

  指定数据处理数量

    operationcount=1000

  指定读取操作数

    readproportion=0.95

  指定更新操作数

    updateproportion=0.05

  参考
<LI><b>fieldcount</b>: the number of fields in a record (default: 10) 
<LI><b>fieldlength</b>: the size of each field (default: 100) 
<LI><b>readallfields</b>: should reads read all fields (true) or just one (false) (default: true) 
<LI><b>readproportion</b>: what proportion of operations should be reads (default: 0.95) 
<LI><b>updateproportion</b>: what proportion of operations should be updates (default: 0.05) 
<LI><b>insertproportion</b>: what proportion of operations should be inserts (default: 0) 
<LI><b>scanproportion</b>: what proportion of operations should be scans (default: 0) 
<LI><b>readmodifywriteproportion</b>: what proportion of operations should be read a record, modify it, write it back (default: 0) 
<LI><b>requestdistribution</b>: what distribution should be used to select the records to operate on - uniform, zipfian or latest (default: uniform) 
<LI><b>maxscanlength</b>: for scans, what is the maximum number of records to scan (default: 1000) 
<LI><b>scanlengthdistribution</b>: for scans, what distribution should be used to choose the number of records to scan, for each scan, between 1 and maxscanlength (default: uniform) 
<LI><b>insertorder</b>: should records be inserted in order by key ("ordered"), or in hashed order ("hashed") (default: hashed)
<LI><b>fieldnameprefix</b>: string prefix for the field name (default: “field”)
文章来自个人专栏
文章 | 订阅
0条评论
0 / 1000
请输入你的评论
0
0