Hello,

I am little bit puzzled my Reading is faster than Writing, should not be other way around? Writing on single OST is about 400MiB/s using Lustre.
If I write directly to raid it brings 1200MiB/s and random 800MiB/s

Could you please advise where to dig or tuneup?

Thanks in advance

Arman.

tuned-admin

My recent IOR test shows:
mpirun -np 4 src/ior  -a POSIX -r -w -b 32g -t 1m -o /lustre/arm2arm/IORFile -O lustreStripeCount=-1 -v -i 3
Summary of all tests:
Operation   Max(MiB)   Min(MiB)  Mean(MiB)     StdDev    Mean(s) Test# #Tasks tPN reps fPP reord reordoff reordrand seed segcnt blksiz xsize aggsize API RefNum
write         343.27     177.23     287.06      77.67  502.04247 0 4 4 3 0 0 1 0 0 1 34359738368 1048576 137438953472 POSIX 0
read          788.68     768.17     776.53       8.79  168.81345 0 4 4 3 0 0 1 0 0 1 34359738368 1048576 137438953472 POSIX 0

the mds-survey shows  good performance:
 thrhi=16 dir_count=16 file_count=200000 mds-survey
Thu Aug  1 12:05:12 CEST 2013 /usr/bin/mds-survey from araid062
mdt 1 file  200000 dir   16 thr   16 create 33506.27 [25998.91,36998.41] lookup 790273.32 [790273.32,790273.32] md_getattr 474972.51 [474972.51,474972.51] setxattr 11483.39 [   0.00,21999.03] destroy 73063.39 [63998.59,63998.59]
done!
 

My hardware is following:
Raid box has a 24 SATA Disks(7200rpm with adaptec 6805controller raid6) ,DDR IB, 32GB Ram, 8core CPU.

RaidBox1: mds/mdt+oss2

            drbd |

RaidBox2: mds/mdt(slave)+oss1

Each box has a sigle port DDR IBs.

oss1, oss2- 34T - used 75%

mdt:    92G       -  used  3%

Software SL6.4, Lustre 2.4

kernel  2.6.32-358.6.2

 tuned-adm active
Current active profile: latency-performance
Service tuned: enabled, running
Service ktune: enabled, running