6:04 a.m.
Dear Andreas, Rick, & Robin
Thank you for your input on this.
Our main motivation to go with OST pools versus multiple file system was to allow
non-savvy end users to simply point to a directory and write to a faster performing disk
as opposed to SATA, in the hope that everything else is aligned well for them to gain the
bandwidth and performance out of our storage. If I understand this correctly our current
storage setup is such that if we were to exclusively use SAS vs SATA we have enough
storage/bandwidth of each type to max out the performance on our DDN controllers below the
12 OSS. Please let me know if there are any flaws in what I say.
On the other hand we wanted to have an easy way to manage the entire file-system as
opposed to multiple files system, and also the ability to scale across when we fill up. To
add to this as you point out savvy users could use all OST's if required.
A follow up question based on your input:
I was hoping to use quota just so we could keep tabs on the usage: Using OST pools, do we
have to rule out the possibility of using quota feature all together ? I was hoping to
accomplish the following, would quota's work at any level of the hierarchy?
/lustre/{sas,sata}/<user>
Please advise.
Thank you, Amit H. Kumar
-----Original Message-----
From: Dilger, Andreas [mailto:andreas.dilger@intel.com]
Sent: Monday, July 21, 2014 1:01 PM
To: Robin Humble; Kumar, Amit
Cc: hpdd-discuss(a)lists.01.org
Subject: Re: [HPDD-discuss] Lustre Pools vs Multiple Lustre File System recommendation
On 2014/07/21, 11:27 AM, "Robin Humble" <rjh+lustre(a)cita.utoronto.ca>
wrote:
On Mon, Jul 21, 2014 at 02:15:31PM +0000, Kumar, Amit wrote:
>Seeking for recommendations on setting up multiple Lustre file system
>"versus" creating multiple pools with exclusive OST's.
>
>We have two sets of disks SAS vs SATA and would like to separate them.
>I have created exclusive OST pools and they seem to work fine, but I
>am looking for recommendations before we go into production with this setup.
>We have 12 OSS and all of the SAS & SATA OST's are evenly spread
>across
>12 OSS's.
>
>Any thoughts on this is greatly appreciated.
I suspect you would get ~2x the iops if you used 2 fs's and 2 MDS's.
if you're already running an active/passive MDS pair then this would
mean no extra hardware.
That depends on where your bottleneck is. Yes, you would increase the metadata
ops/second, but you would only get 1/2 of your IO bandwidth for each filesystem if you
split the OSTs across two filesystems.
I think the one major limitation to using OST pools today is that they are
"advisory" only, and savvy users could specify any OST pool that they want.
That may be ok, but it depends on your users and environment.
We've discussed implementing usage quotas/limits on OST pools, but this has not been
implemented yet.
Cheers, Andreas
--
Andreas Dilger
Lustre Software Architect
Intel High Performance Data Division