4:40 a.m.
Hi,
I was trying to run ost-survey on this environment
mds, oss: lustre 2.4.2 (CentOS 6.5 with lustre patched kernel RPM)
client: lustre 2.4.2 (CentOS 6.5 patchless client, lustre iokit 1.4.0.1)
command line: ost-survey -s 100 /lustre
but the script hangs each time, chewing all the CPU in the attempt to
echo 0 in /proc/fs/lustre/llite/*/max_cached_mb . This is due to the
subroutine cache_off in the ost-survey script.
But while in a lustre client 2.1.6 the max_cached_mb appears to me as
a single number
# cat /proc/fs/lustre/llite/*/max_cached_mb
18114
on a 2.4.2 client is different
# cat /proc/fs/lustre/llite/*/max_cached_mb
users: 2
max_cached_mb: 96766
used_mb: 21806
unused_mb: 74960
reclaim_count: 0
Am I missing a new version of ost-survey or should I simply get rid of
those dangerous lines in the script?
Thanks,
Francesco
5:09 a.m.
New subject: llapi_file_get_stripe arguments
I am using the llapi_file_get_stripe() function.
The second arguement is
struct lov_user_md_v1 {
__u32 lmm_magic;
__u32 lmm_pattern;
__u64 lmm_object_id;
__u64 lmm_object_seq;
__u32 lmm_stripe_size;
__u16 lmm_stripe_count;
__u16 lmm_stripe_offset;
struct lov_user_ost_data_v1 lmm_objects[0];
}
How do I indicate to the kernel how many elements are in the lmn_objects[] array?
Or, How do I know how large to make the array?
What is the largest number of OST's that a file can be striped across?
Thanks
John
5:27 a.m.
New subject: llapi_file_get_stripe arguments
Ø What is the largest number of OST's that a file can be striped across?
Hi John,
Starting with Lustre version 2.2, the largest possible stripe count for a single file is
2000 OSTs.
Prior to that it was 160 OSTs.
[Description: cid:image001.gif@01C9FE3D.1D8A68C0]
Dr. Brett Lee, Solutions Architect
High Performance Data Division, Intel
+1.303.625.3595
From: HPDD-discuss [mailto:hpdd-discuss-bounces@lists.01.org] On Behalf Of John Bauer
Sent: Thursday, March 13, 2014 12:09 PM
To: hpdd-discuss(a)lists.01.org
Subject: [HPDD-discuss] llapi_file_get_stripe arguments
I am using the llapi_file_get_stripe() function.
The second arguement is
struct lov_user_md_v1 {
__u32 lmm_magic;
__u32 lmm_pattern;
__u64 lmm_object_id;
__u64 lmm_object_seq;
__u32 lmm_stripe_size;
__u16 lmm_stripe_count;
__u16 lmm_stripe_offset;
struct lov_user_ost_data_v1 lmm_objects[0];
}
How do I indicate to the kernel how many elements are in the lmn_objects[] array?
Or, How do I know how large to make the array?
What is the largest number of OST's that a file can be striped across?
Thanks
John
8:40 a.m.
On 2014/03/13, 11:40 AM, "Francesco De Giorgi"
<francesco.degiorgi(a)exact-lab.it> wrote:
Hi,
I was trying to run ost-survey on this environment
mds, oss: lustre 2.4.2 (CentOS 6.5 with lustre patched kernel RPM)
client: lustre 2.4.2 (CentOS 6.5 patchless client, lustre iokit 1.4.0.1)
command line: ost-survey -s 100 /lustre
but the script hangs each time, chewing all the CPU in the attempt to
echo 0 in /proc/fs/lustre/llite/*/max_cached_mb . This is due to the
subroutine cache_off in the ost-survey script.
That is probably a bug in the script, and also in the kernel to allow this
to be set. It doesn't make sense to prevent any pages to be cached on the
client. It probably makes sense to require at least a few MB of cache
space on the client.
If some site wants minimal memory usage at the expense of performance,
that is OK, but we shouldn't allow them to break the system completely.
But while in a lustre client 2.1.6 the max_cached_mb appears to me as
a single number
# cat /proc/fs/lustre/llite/*/max_cached_mb
18114
on a 2.4.2 client is different
# cat /proc/fs/lustre/llite/*/max_cached_mb
users: 2
max_cached_mb: 96766
used_mb: 21806
unused_mb: 74960
reclaim_count: 0
Am I missing a new version of ost-survey or should I simply get rid of
those dangerous lines in the script?
It looks like the script has not been fixed to work with the new output.
Could you please file a bug at https://jira.hpdd.intel.com/ with the above
details so that it can be fixed.
Cheers, Andreas
--
Andreas Dilger
Lustre Software Architect
Intel High Performance Data Division
10:56 p.m.
On Thu, Mar 13, 2014 at 10:40 PM, Dilger, Andreas
<andreas.dilger(a)intel.com> wrote:
[...]
It looks like the script has not been fixed to work with the new output.
Could you please file a bug at https://jira.hpdd.intel.com/ with the above
details so that it can be fixed.
Cheers, Andreas
--
Andreas Dilger
Lustre Software Architect
Intel High Performance Data Division
LU-4768
https://jira.hpdd.intel.com/browse/LU-4768
Thanks,
Francesco
--
Francesco De Giorgi
eXact lab s.r.l.
+39 329 1529493
www.exact-lab.it
4:24 a.m.
New subject: llapi_file_get_stripe arguments
On Thu, 2014-03-13 at 13:09 -0500, John Bauer wrote:
I am using the llapi_file_get_stripe() function.
The second arguement is
struct lov_user_md_v1 {
__u32 lmm_magic;
__u32 lmm_pattern;
__u64 lmm_object_id;
__u64 lmm_object_seq;
__u32 lmm_stripe_size;
__u16 lmm_stripe_count;
__u16 lmm_stripe_offset;
struct lov_user_ost_data_v1 lmm_objects[0];
}
How do I indicate to the kernel how many elements are in the lmn_objects[] array?
Or, How do I know how large to make the array?
Does the example here help?:
http://build.whamcloud.com/job/lustre-manual/lastSuccessfulBuild/artifact...
What is the largest number of OST's that a file can be striped
across?
For a related discussion, you can see this ticket:
https://jira.hpdd.intel.com/browse/LU-1707
best regards,
Richard
--
Richard.Henwood(a)intel.com
High Performance Data Division
3110
days inactive
3111
days old
5 comments
5 participants
participants (5)
-
Dilger, Andreas -
Francesco De Giorgi -
Henwood, Richard -
John Bauer -
Lee, Brett