[HPDD-discuss] OSC and buffer cache throttling

All, I'll start with the question: Is there a Lustre config parameter that throttles the amount of system memory that a given OSC can use when reading a file? I am running an iozone job, single process, single thread. Writes an 80GB file twice, reads forwards twice, reads backwards twice. This is run with varying stripe count values ( 2,3,4,5,6,7,8,12 and 16 ) with 1MB stripe. While each job is running I monitor the "Cached" value in /proc/meminfo. That "Cached" value is plotted against time for the duration of the runs. The jobs are all run independently. The client node has 128 GB of memory, so the file should easily fit in the system buffers. The filesystem has 16 OSTs. As indicated in the following plot, the maximum value for "Cached" when stripe_count=2 is 17GB stripe_count=3 is 25GB stripe_count=4 is 33GB stripe_count=5 is 42GB stripe_count=6 is 50GB stripe_count=7 is 59GB stripe_count=8,12,16 are all 63GB What is limiting the system buffer utilization? I have been grep'ing around in the Lustre documentation and don't find many references to system cache or buffer cache. And none that explain this. Just for completeness, here is a plot of how the file is being written,read forward,read backward where it is easy to see when the data is coming from the system buffers. This is for OST=3 Thanks John