Re: [HPDD-discuss] Disk provisioning for Lustre
On 04/04/2013 05:57 AM, Thomas Roth wrote:
Hi,
there were several discussions of the topic on the Lustre mailing lists, e.g.
"[Lustre-discuss]
clarification on mkfs.lustre options, 30.7.2010", "Re: [Lustre-discuss] mkfs
options/tuning for RAID
based OSTs, 20.10.2010"
As far as I understand, everything derives from Lustre's basic 1MB rpc size. In the
end, the number of
data disks in a RAID should be a power of 2, so Raid6 = 4+2, 8+2, etc.
According to this, your Raids have two disks too many...
Also, in the quoted discussions, Andreas Dilger made a strong point about why
partitioning is evil for
performance.
In fact I have just tested a number of new servers which came with Raid 6, each 10 disks,
but a "strip
size" (LSI-controller-speak) of 512kB. I have since recreated the Raids, changing
this strip size to
128kB (8 x 128 kB = 1MB), and the results as seen by ost-survey ( from lustre-iokit) show
three times
more throughput for 100MB files. (A bit too good for my taste, but at least it points to
the right
direction.)
Cheers,
Thomas
I found some of those discussions last night after I posted.
The "power of 2" aspect has finally sunk in.
With 12 disks, I can do two sets of RAID6 4+2 or one set of 8+2 and have
two hot spares. From a space efficiency standpoint, it looks like I
end up with a similar amount of usable disk space. The 8+2 with the hot
spares would seem to be the better choice.
Is there any reason I would pick the two sets of RAID6 4+2 over the one
RAID6 8+2? The latter seems to be the better choice.
--
Ray Muno
Information Technology Supervisor
University of Minnesota
Aerospace Engineering and Mechanics Mechanical Engineering
110 Union St. S.E. 111 Church Street SE
Minneapolis, MN 55455 Minneapolis, MN 55455