読者です 読者をやめる 読者になる 読者になる

ReadyNAS Ultra4 Plusに触ってみた その4

ReadyNAS benchmark

ReadyNASにEnable Root SSHを導入してリモートからSSHで接続していろいろテスト。
Enable Root SSHの導入自体はダウンロードしたbinファイルをReadyNAS上でインストールして再起動するだけなので詳細は省略。

ReadyNASへのSSH接続

再起動後にSSH接続してみると見慣れた画面が出てくる。

# uname -a
Linux nas-C1-28-9A 2.6.37.6.RNx86_64.2.1 #1 SMP Mon Aug 15 16:19:41 PDT 2011 x86_64 GNU/Linux

# cat /etc/debian_version
4.0

どうやらDebian 4.0 etchがベースらしい。

fioのインストール

fioを導入すべく、apt-get updateを実行するとエラーが出る。調べてみると4.2.19固有の不具合の模様。
必要なパッケージをインストールしてからfioを導入。

# apt-get install libaio-dev make gcc build-essential
# wget http://brick.kernel.dk/snaps/fio-2.0.5.tar.gz
# tar zxvf fio-2.0.5.tar.gz
# cd fio
# make
# make install

その3でiSCSIのベンチマークを行った時と同様にテスト。ただし、今回はdirectory=/cにしている。詳細な理由は次回以降に記述するが、そこがNASのユーザ領域になるからとしておく。
…書くことがどんどん増えていく。

シーケンシャルリード

seqread: (g=0): rw=read, bs=1M-1M/1M-1M, ioengine=sync, iodepth=1
fio 2.0.5
Starting 1 process

seqread: (groupid=0, jobs=1): err= 0: pid=5124
  read : io=1024.0MB, bw=367715KB/s, iops=359 , runt= 14258msec
    clat (msec): min=1 , max=93 , avg= 2.77, stdev= 2.26
     lat (msec): min=1 , max=93 , avg= 2.77, stdev= 2.26
    clat percentiles (usec):
     |  1.00th=[ 2008],  5.00th=[ 2008], 10.00th=[ 2024], 20.00th=[ 2064],
     | 30.00th=[ 2128], 40.00th=[ 2192], 50.00th=[ 2352], 60.00th=[ 2448],
     | 70.00th=[ 2640], 80.00th=[ 3056], 90.00th=[ 3536], 95.00th=[ 5280],
     | 99.00th=[ 6752], 99.50th=[11584], 99.90th=[22912]
    bw (KB/s)  : min=276866, max=397312, per=99.85%, avg=367157.32, stdev=28419.65
    lat (msec) : 2=0.59%, 4=92.13%, 10=6.66%, 20=0.49%, 50=0.10%
    lat (msec) : 100=0.04%
  cpu          : usr=0.60%, sys=18.26%, ctx=5123, majf=0, minf=0
  IO depths    : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     issued    : total=r=5120/w=0/d=0, short=r=0/w=0/d=0

Run status group 0 (all jobs):
   READ: io=5120.0MB, aggrb=367714KB/s, minb=376540KB/s, maxb=376540KB/s, mint=14258msec, maxt=14258msec

Disk stats (read/write):
    dm-0: ios=81424/0, merge=0/0, ticks=113246/0, in_queue=113244, util=94.77%, aggrios=81920/0, aggrmerge=0/0, aggrticks=0/0, aggrin_queue=0, aggrutil=0.00%
    md2: ios=81920/0, merge=0/0, ticks=0/0, in_queue=0, util=0.00%, aggrios=10240/6, aggrmerge=10239/3, aggrticks=14154/83, aggrin_queue=14235, aggrutil=68.40%
  sdb: ios=10241/6, merge=10239/3, ticks=13600/100, in_queue=13695, util=64.53%
  sdc: ios=10240/6, merge=10240/3, ticks=13474/85, in_queue=13557, util=62.20%
  sdd: ios=10240/6, merge=10240/3, ticks=15542/66, in_queue=15607, util=68.40%
  sda: ios=10240/6, merge=10240/3, ticks=14002/83, in_queue=14082, util=65.66%

シーケンシャルライト

seqwrite: (g=0): rw=write, bs=1M-1M/1M-1M, ioengine=sync, iodepth=1
fio 2.0.5
Starting 1 process
seqwrite: Laying out IO file(s) (1 file(s) / 1024MB)
Jobs: 1 (f=1): [W] [100.0% done] [0K/8355K /s] [0 /7  iops] [eta 00m:00s]
seqwrite: (groupid=0, jobs=1): err= 0: pid=8990
  write: io=5120.0MB, bw=36115KB/s, iops=35 , runt=145173msec
    clat (msec): min=13 , max=733 , avg=28.26, stdev=36.18
     lat (msec): min=13 , max=733 , avg=28.35, stdev=36.18
    clat percentiles (msec):
     |  1.00th=[   14],  5.00th=[   15], 10.00th=[   15], 20.00th=[   15],
     | 30.00th=[   15], 40.00th=[   16], 50.00th=[   17], 60.00th=[   21],
     | 70.00th=[   26], 80.00th=[   33], 90.00th=[   48], 95.00th=[   76],
     | 99.00th=[  165], 99.50th=[  223], 99.90th=[  469]
    bw (KB/s)  : min= 2409, max=63109, per=100.00%, avg=37125.47, stdev=13231.65
    lat (msec) : 20=59.77%, 50=30.78%, 100=6.23%, 250=2.75%, 500=0.37%
    lat (msec) : 750=0.10%
  cpu          : usr=0.31%, sys=0.44%, ctx=5179, majf=0, minf=25
  IO depths    : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     issued    : total=r=0/w=5120/d=0, short=r=0/w=0/d=0

Run status group 0 (all jobs):
  WRITE: io=5120.0MB, aggrb=36114KB/s, minb=36981KB/s, maxb=36981KB/s, mint=145173msec, maxt=145173msec

Disk stats (read/write):
  sda: ios=2/25363, merge=0/1425, ticks=37/778774, in_queue=778822, util=99.51%

ランダムリード 512k

randread512k: (g=0): rw=randread, bs=512K-512K/512K-512K, ioengine=sync, iodepth=1
fio 2.0.5
Starting 1 process
randread512k: Laying out IO file(s) (1 file(s) / 1024MB)

randread512k: (groupid=0, jobs=1): err= 0: pid=5644
  read : io=1024.0MB, bw=44493KB/s, iops=86 , runt=117835msec
    clat (msec): min=1 , max=103 , avg=11.49, stdev= 4.02
     lat (msec): min=1 , max=103 , avg=11.49, stdev= 4.02
    clat percentiles (usec):
     |  1.00th=[ 1592],  5.00th=[ 6816], 10.00th=[ 8256], 20.00th=[ 9792],
     | 30.00th=[10560], 40.00th=[11200], 50.00th=[11712], 60.00th=[12096],
     | 70.00th=[12480], 80.00th=[12864], 90.00th=[13376], 95.00th=[13888],
     | 99.00th=[24704], 99.50th=[33536], 99.90th=[57088]
    bw (KB/s)  : min=27592, max=50996, per=99.92%, avg=44458.70, stdev=3088.60
    lat (msec) : 2=1.56%, 4=0.57%, 10=20.76%, 20=75.22%, 50=1.73%
    lat (msec) : 100=0.15%, 250=0.01%
  cpu          : usr=0.13%, sys=2.56%, ctx=10254, majf=0, minf=0
  IO depths    : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     issued    : total=r=10240/w=0/d=0, short=r=0/w=0/d=0

Run status group 0 (all jobs):
   READ: io=5120.0MB, aggrb=44493KB/s, minb=45561KB/s, maxb=45561KB/s, mint=117835msec, maxt=117835msec

Disk stats (read/write):
    dm-0: ios=81920/18, merge=0/0, ticks=626863/770, in_queue=627693, util=99.26%, aggrios=81920/18, aggrmerge=0/0, aggrticks=0/0, aggrin_queue=0, aggrutil=0.00%
    md2: ios=81920/18, merge=0/0, ticks=0/0, in_queue=0, util=0.00%, aggrios=13670/82, aggrmerge=6829/45, aggrticks=105204/1002, aggrin_queue=106187, aggrutil=69.97%
  sdb: ios=13672/82, merge=6832/41, ticks=105787/1013, in_queue=106793, util=68.88%
  sdc: ios=13670/79, merge=6838/41, ticks=107480/898, in_queue=108372, util=69.97%
  sdd: ios=13668/84, merge=6824/50, ticks=102860/970, in_queue=103828, util=67.47%
  sda: ios=13673/85, merge=6823/50, ticks=104692/1130, in_queue=105758, util=68.65%

ランダムライト 512k

randwrite512k: (g=0): rw=randwrite, bs=512K-512K/512K-512K, ioengine=sync, iodepth=1
fio 2.0.5
Starting 1 process
randwrite512k: Laying out IO file(s) (1 file(s) / 1024MB)

randwrite512k: (groupid=0, jobs=1): err= 0: pid=6256
  write: io=1024.0MB, bw=20329KB/s, iops=39 , runt=257902msec
    clat (msec): min=3 , max=465 , avg=25.01, stdev=28.90
     lat (msec): min=3 , max=465 , avg=25.17, stdev=28.90
    clat percentiles (msec):
     |  1.00th=[    5],  5.00th=[   11], 10.00th=[   13], 20.00th=[   15],
     | 30.00th=[   16], 40.00th=[   18], 50.00th=[   21], 60.00th=[   24],
     | 70.00th=[   28], 80.00th=[   32], 90.00th=[   39], 95.00th=[   45],
     | 99.00th=[   73], 99.50th=[  310], 99.90th=[  404]
    bw (KB/s)  : min= 4071, max=33657, per=100.00%, avg=20658.85, stdev=5288.02
    lat (msec) : 4=0.87%, 10=2.89%, 20=45.86%, 50=47.28%, 100=2.34%
    lat (msec) : 250=0.21%, 500=0.55%
  cpu          : usr=0.66%, sys=3.84%, ctx=10327, majf=0, minf=0
  IO depths    : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     issued    : total=r=0/w=0/d=10240, short=r=0/w=0/d=0

Run status group 0 (all jobs):
  WRITE: io=5120.0MB, aggrb=20328KB/s, minb=20816KB/s, maxb=20816KB/s, mint=257902msec, maxt=257902msec

Disk stats (read/write):
    dm-0: ios=1/82214, merge=0/0, ticks=8/1643859, in_queue=1643906, util=99.06%, aggrios=1/82312, aggrmerge=0/0, aggrticks=0/0, aggrin_queue=0, aggrutil=0.00%
    md2: ios=1/82312, merge=0/0, ticks=0/0, in_queue=0, util=0.00%, aggrios=17605/31724, aggrmerge=235079/433420, aggrticks=206131/103203, aggrin_queue=309479, aggrutil=67.46%
  sdb: ios=17726/33928, merge=239823/459607, ticks=202144/109413, in_queue=311921, util=64.01%
  sdc: ios=16193/34139, merge=232567/459013, ticks=188005/109286, in_queue=297401, util=61.48%
  sdd: ios=19809/30159, merge=278272/406805, ticks=243615/101061, in_queue=344717, util=67.46%
  sda: ios=16692/28673, merge=189656/408256, ticks=190762/93055, in_queue=283877, util=61.36%

ランダムリード4k

# fio randread4k.fio
randread4k: (g=0): rw=randread, bs=4K-4K/4K-4K, ioengine=sync, iodepth=1
fio 2.0.5
Starting 1 process
randread4k: Laying out IO file(s) (1 file(s) / 100MB)
Jobs: 1 (f=1): [r] [100.0% done] [2213K/0K /s] [540 /0  iops] [eta 00m:00s]
randread4k: (groupid=0, jobs=1): err= 0: pid=13218
  read : io=307200KB, bw=1796.1KB/s, iops=449 , runt=170954msec
    clat (usec): min=120 , max=92009 , avg=2215.24, stdev=3882.82
     lat (usec): min=122 , max=92011 , avg=2216.65, stdev=3882.82
    clat percentiles (usec):
     |  1.00th=[  193],  5.00th=[  215], 10.00th=[  245], 20.00th=[  290],
     | 30.00th=[  338], 40.00th=[  398], 50.00th=[  466], 60.00th=[  556],
     | 70.00th=[  724], 80.00th=[ 3952], 90.00th=[ 8256], 95.00th=[10432],
     | 99.00th=[15040], 99.50th=[18560], 99.90th=[28032]
    bw (KB/s)  : min=  761, max= 2563, per=99.47%, avg=1786.48, stdev=258.32
    lat (usec) : 250=11.27%, 500=42.49%, 750=17.10%, 1000=3.20%
    lat (msec) : 2=1.71%, 4=4.36%, 10=13.92%, 20=5.58%, 50=0.32%
    lat (msec) : 100=0.05%
  cpu          : usr=0.51%, sys=2.55%, ctx=76846, majf=0, minf=0
  IO depths    : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     issued    : total=r=76800/w=0/d=0, short=r=0/w=0/d=0

Run status group 0 (all jobs):
   READ: io=307200KB, aggrb=1796KB/s, minb=1840KB/s, maxb=1840KB/s, mint=170954msec, maxt=170954msec

Disk stats (read/write):
    dm-0: ios=76648/10, merge=0/0, ticks=167077/210, in_queue=167286, util=98.07%, aggrios=76800/10, aggrmerge=0/0, aggrticks=0/0, aggrin_queue=0, aggrutil=0.00%
    md2: ios=76800/10, merge=0/0, ticks=0/0, in_queue=0, util=0.00%, aggrios=19203/192, aggrmerge=0/81, aggrticks=41745/1314, aggrin_queue=43047, aggrutil=25.48%
  sdb: ios=19202/193, merge=1/82, ticks=40379/1285, in_queue=41656, util=24.17%
  sdc: ios=19207/188, merge=1/82, ticks=42548/1282, in_queue=43820, util=25.48%
  sdd: ios=19202/192, merge=1/83, ticks=41647/1334, in_queue=42964, util=24.91%
  sda: ios=19203/195, merge=0/80, ticks=42406/1357, in_queue=43751, util=25.36%||<

** ランダムライト4k
>||
# fio randwrite4k.fio
randwrite4k: (g=0): rw=randwrite, bs=4K-4K/4K-4K, ioengine=sync, iodepth=1
fio 2.0.5
Starting 1 process
randwrite4k: Laying out IO file(s) (1 file(s) / 100MB)
Jobs: 1 (f=1): [w] [100.0% done] [0K/1091K /s] [0 /266  iops] [eta 00m:00s]
randwrite4k: (groupid=0, jobs=1): err= 0: pid=11855
  write: io=307200KB, bw=960935 B/s, iops=234 , runt=327361msec
    clat (usec): min=274 , max=211192 , avg=4249.68, stdev=10041.85
     lat (usec): min=276 , max=211194 , avg=4251.68, stdev=10041.86
    clat percentiles (usec):
     |  1.00th=[  294],  5.00th=[  302], 10.00th=[  310], 20.00th=[  322],
     | 30.00th=[  334], 40.00th=[  354], 50.00th=[  374], 60.00th=[  406],
     | 70.00th=[  502], 80.00th=[ 4448], 90.00th=[13248], 95.00th=[26496],
     | 99.00th=[48384], 99.50th=[55552], 99.90th=[78336]
    bw (KB/s)  : min=  333, max= 1733, per=99.85%, avg=936.58, stdev=286.28
    lat (usec) : 500=69.96%, 750=5.38%, 1000=2.72%
    lat (msec) : 2=0.90%, 4=0.81%, 10=6.17%, 20=6.97%, 50=6.23%
    lat (msec) : 100=0.83%, 250=0.03%
  cpu          : usr=0.30%, sys=1.70%, ctx=76944, majf=0, minf=0
  IO depths    : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     issued    : total=r=0/w=0/d=76800, short=r=0/w=0/d=0

Run status group 0 (all jobs):
  WRITE: io=307200KB, aggrb=938KB/s, minb=960KB/s, maxb=960KB/s, mint=327361msec, maxt=327361msec

Disk stats (read/write):
    dm-0: ios=1/77948, merge=0/0, ticks=6/356186, in_queue=356218, util=98.69%, aggrios=1/78121, aggrmerge=0/0, aggrticks=0/0, aggrin_queue=0, aggrutil=0.00%
    md2: ios=1/78121, merge=0/0, ticks=0/0, in_queue=0, util=0.00%, aggrios=8443/38992, aggrmerge=151/564, aggrticks=98013/20708, aggrin_queue=118693, aggrutil=36.32%
  sdb: ios=8422/39071, merge=148/565, ticks=102467/20690, in_queue=123128, util=36.32%
  sdc: ios=8264/38984, merge=149/558, ticks=92527/20348, in_queue=112847, util=33.17%
  sdd: ios=8715/38831, merge=151/576, ticks=100732/21697, in_queue=122393, util=35.94%
  sda: ios=8374/39085, merge=159/557, ticks=96328/20098, in_queue=116404, util=34.34%

多重化した項目も同様に計測したが、詳細な結果は省略する。需要があれば掲載。

先のiSCSIの結果と同様に表にすると以下のようになる。

TypeiSCSI (ext3)X-RAID2 (ext4)
Speed
(MB/s)
IOPSSpeed
(MB/s)
IOPS
seq read66.264367.7359
seq write36.13554.353
seq read*447.7-152.9-
seq write*437.1-78.5-
rand read 512k11.32244.586
rand write 512k34.36620.339
rand read 4k10.826991.8452
rand write 4k8.420871.0234
rand read 4k*3217.0-7.0-
rand write 4k*326.9-1.2-

(※ ランダムリード4kの結果を張り間違えてたので修正@3/21)

結果はiSCSIと大きく異なるもので、iSCSIでシーケンシャル性能がどれだけ生かされてないかというのがよく分かった。

次回はX-RAID2がどうなっているかについて。