testfio: (g=0): rw=randrw, bs=(R) 16.0KiB-16.0KiB, (W) 16.0KiB-16.0KiB, (T) 16.0KiB-16.0KiB, ioengine=libaio, iodepth=32
...
fio-3.7
Starting 10 threads
Jobs: 8 (f=8): [m(4),_(1),m(3),_(1),m(1)][a123%][r=a123 MiB/s,w=a123 MiB/s][r=a123k,w=a123k IOPS][eta 00m:03s]
testfio: (groupid=0, jobs=10): err= 0: pid=3335: Thu Sep xx xx:xx:xx 20xx
   read: IOPS=a123 k, BW= a123 MiB/s (a123 MB/s)(a123 GiB/a123 msec)
    slat (usec): min=a123, max=a123, avg=a123, stdev=a123
    clat (usec): min=a123, max=a123, avg=a123, stdev=a123
     lat (usec): min=a123, max=a123, avg=a123, stdev=a123
    clat percentiles (usec):
     |  1.00th=[   a123],  5.00th=[  a123], 10.00th=[  a123], 20.00th=[  a123],
     | 30.00th=[  a123], 40.00th=[  a123], 50.00th=[  a123], 60.00th=[  a123],
     | 70.00th=[  a123], 80.00th=[ a123], 90.00th=[ a123], 95.00th=[ a123],
     | 99.00th=[ a123], 99.50th=[ a123], 99.90th=[a123], 99.95th=[a123],
     | 99.99th=[a123]
   bw (  KiB/s): min=  a123, max=a123, per=a123%, avg=41984.49a123, stdev=a123, samples=a123
   iops        : min=   a123, max= a123, avg=a123, stdev=a123, samples=a123
  write: IOPS=a123 k, BW=a123 MiB/s (a123 MB/s)(a123 MiB/a123 msec)
    slat (usec): min=a123, max=a123, avg=a123, stdev=a123
    clat (usec): min=a123, max=a123, avg=a123, stdev=a123
     lat (usec): min=a123, max=a123, avg=a123, stdev=a123
    clat percentiles (usec):
     |  1.00th=[   a123],  5.00th=[  a123], 10.00th=[  a123], 20.00th=[  a123],
     | 30.00th=[  a123], 40.00th=[  a123], 50.00th=[  a123], 60.00th=[  a123],
     | 70.00th=[  a123], 80.00th=[ a123], 90.00th=[ a123], 95.00th=[ a123],
     | 99.00th=[ a123], 99.50th=[ a123], 99.90th=[a123], 99.95th=[a123],
     | 99.99th=[a123]
   bw (  KiB/s): min=  a123, max=a123, per=a123%, avg=a123, stdev=a123, samples=a123
   iops        : min=   a123, max= a123, avg=a123, stdev=a123, samples=a123
  lat (usec)   : 100=a123%, 250=a123%, 500=a123%, 750=a123%, 1000=a123%
  lat (msec)   : 2=a123%, 4=a123%, 10=a123%, 20=a123%, 50=a123%
  lat (msec)   : 100=a123%, 250=a123%, 500=a123%
  cpu          : usr=a123%, sys=a123%, ctx=a123, majf=a123, minf=a123
  IO depths    : 1=a123%, 2=a123%, 4=a123%, 8=a123%, 16=a123%, 32=a123%, >=64=a123%
     submit    : 0=a123%, 4=a123%, 8=a123%, 16=a123%, 32=a123%, 64=a123%, >=64=a123%
     complete  : 0=a123%, 4=a123%, 8=a123%, 16=a123%, 32=a123%, 64=a123%, >=64=a123%
     issued rwts: total=a123,a123,0,0 short=0,0,0,0 dropped=0,0,0,0
     latency   : target=0, window=0, percentile=100.00%, depth=32

Run status group 0 (all jobs):
   READ: bw=a123MiB/s (a123MB/s), a123MiB/s-a123MiB/s (a123MB/s-a123MB/s), io=a123GiB (a123 GB), run=a123-a123msec
  WRITE: bw=a123MiB/s (a123 MB/s), a123 MiB/s-a123 MiB/s (a123 MB/s-a123 MB/s), io=a123 MiB (a123MB), run=a123-a123 msec

Disk stats (read/write):
  vdb: ios=a123/a123, merge=0/0, ticks=a123/a123, in_queue=a123, util=a123%

1、

Jobs： 8 表示当前执行的I/O线程数为8
（f=8）表示当前打开的文件数量为8
[a123%]表示当前执行进度
[r=a123 MiB/s,w=a123 MiB/s] 该组中括号，r表示随机读，w表示随机写，若为大写字母R或W则代表顺序读顺序写

IOPS：每秒执行的IO个数，是评估磁盘性能的重要指标之一；
BW：dai在实际的性能测试过程中，小io会更关注IOPS,大IO会更关注带宽
(a123 GiB/a123 msec)分别代表 此次执行段总IO和此次线程的运行时间

I/O时延包括三种，slat  clat 和lat
其中，slat代表某个I/O提交到内核所用时间，即 fio submit延迟，后面数据分别表示fio submit延迟的最小值、最大值、平均值和标准差。slat (usec)括号中为单位，本例中为nsex
clat代表完成某个I/O所用时间（从I/O提交到内核到io完成返回），即 fio complete延迟，其余参数同slat。
lat= slat + clat，代表从请求到内核，再到内核完成该I/O所用时间

clat percentiles： clat的置信级，1.00th=[   a123]表示有1% I/O平均clat是a123,其他数据同理

bw:基于样本（samples）的带宽统计 
iops：基于样本（samples）的带宽统计

CPU使用率。
usr:用户
sys:系统时间使用率
ctx: 运行过程中上下文切换次数
majf:主要页面错误数量
minf：次要页面错误数量

IO队列的深度统计情况。可以理解为前闭后开的区间，例如2=a123%，表示[2,4)也是就是2到3的深度