Re: [HPDD-discuss] Disk provisioning for Lustre
Hi,
there were several discussions of the topic on the Lustre mailing lists, e.g.
"[Lustre-discuss]
clarification on mkfs.lustre options, 30.7.2010", "Re: [Lustre-discuss] mkfs
options/tuning for RAID
based OSTs, 20.10.2010"
As far as I understand, everything derives from Lustre's basic 1MB rpc size. In the
end, the number of
data disks in a RAID should be a power of 2, so Raid6 = 4+2, 8+2, etc.
According to this, your Raids have two disks too many...
Also, in the quoted discussions, Andreas Dilger made a strong point about why partitioning
is evil for
performance.
In fact I have just tested a number of new servers which came with Raid 6, each 10 disks,
but a "strip
size" (LSI-controller-speak) of 512kB. I have since recreated the Raids, changing
this strip size to
128kB (8 x 128 kB = 1MB), and the results as seen by ost-survey ( from lustre-iokit) show
three times
more throughput for 100MB files. (A bit too good for my taste, but at least it points to
the right
direction.)
Cheers,
Thomas
On 03.04.2013 21:34, Ray Muno wrote:
I am setting up a small Lustre file system to use with our clusters.
It is based on Dell R415 (MDS)
and Dell R515's for the OST's.
Each R515 has 14 disks. Two are configured as RAID 1 for the OS. The other 12 are 2TB
Near Line SAS,
currently configured as a single RAID 6 array. There are four R515's as OST's.
They are connected to a DDR2 Infiniband network, along with all the cluster nodes.
All servers are currently set up with Lustre 2.3.
I am looking for advice on how the RAID setup should be configured in terms of stripe
size, read
caching, read/write policy, etc.
Right now I have a test file system configured with each OSS running with a single OST
configured from
the 12 disk as RAID 6. Is there an advantage to partitioning these in to smaller virtual
disks and
having multiple OST's on each OSS?