4:27 a.m.
John,
I don't have anything to add at the moment, but I am watching your
explorations with interest. Thanks for sharing this.
One question - The blue and red lines coming up the graph... What are
those? (Particularly, the one which peaks and then heads back down?)
- Patrick
On 02/05/2015 07:46 AM, John Bauer wrote:
Richard, Patrick
The more I look at this, the more bizarre it gets. Now when I run
this iozone test, I also track *cached_mb* for each OSC. This plot
has the file position activity plot overlayed
with the value of cached_mb for the 16 OST's that the file was
stripped across. Things are predictable until the size of the file
being written exceeds the amount of memory that
Lustre can use for caching ( during the first write). After that, the
competition for buffer memory by the OSC's starts. Further comment on
the plot image.
I still have not determined why *cached_mb* for any OSC never exceeds
8GB in the test cases where I stripe across 2,3,4,5,6,or 7 OSTs. In
those cases the sum of the cached_mb for
the used OSTs never exceeds the 50% of system memory limit.
John
On 2/3/2015 4:47 PM, Mohr Jr, Richard Frank (Rick Mohr) wrote:
>> On Feb 3, 2015, at 3:58 PM, Patrick Farrell<paf(a)cray.com> wrote:
>>
>> Interesting that Rick's seeing 3/4 on his system. The limit looks to be <
512MB, if I'm reading correctly.
> I glanced at the Lustre source for my 2.5.3 client and found this:
>
> pages = si.totalram - si.totalhigh;
> if (pages >> (20 - PAGE_CACHE_SHIFT) < 512) {
> lru_page_max = pages / 2;
> } else {
> lru_page_max = (pages / 4) * 3;
> }
>
> The way I am reading this is that if the system has <512MB of memory, the
lru_page_max is 1/2 the system RAM. Otherwise, it will be 3/4 of the system RAM.
>
> --
> Rick Mohr
> Senior HPC System Administrator
> National Institute for Computational Sciences
> http://www.nics.tennessee.edu
>
> _______________________________________________
> HPDD-discuss mailing list
> HPDD-discuss(a)lists.01.org
> https://lists.01.org/mailman/listinfo/hpdd-discuss
