4 a.m.
On 2013/10/17 5:34 AM, "Olli Lounela" <olli.lounela(a)helsinki.fi> wrote:
Hi,
We run four-node Lustre 2.3, and I needed to both change hardware
under MGS/MDS and reassign an OSS ip. Just the same, I added a brand
new 10GE network to the system, which was the reason for MDS hardware
change.
Note that in Lustre 2.4 there is a "lctl replace_nids" command that
allows you to change the NIDs without running --writeconf. That doesn't
help you now, but possibly in the future.
I ran tunefs.lustre --writeconf as per chapter 14.4 in Lustre Manual,
and everything mounts fine. Log regeneration apparently works, since
it seems to do something, but exceedingly slowly. Disks show all but
no activity, CPU utilization is zero across the board, and memory
should be no issue. I believe it works, but currently it seems the
1,5*10^9 files (some 55 TiB of data) won't be indexed in a week. My
boss isn't happy when I can't even predict how long this will take, or
even say for sure that it really works.
The --writeconf information is at most a few kB and should only take
seconds to complete. What "reindexing" operation are you referencing?
It should be possible to mount the filesystem immediately (MGS first,
then MDS and OSSes) after running --writeconf.
You didn't really explain what is preventing you from using the filesystem,
since you said it mounted properly?
Two questions: is there a way to know how fast it is progressing
and/or where it is at, or even that it really works, and is there a
way to speed up whatever is slowing it down? Seems all diagnostic
/proc entries have been removed from 2.3. I have tried mounting the
Lustre partitions with -o nobarrier (yes, I know it's dangerous, but
I'd really need to speed things up) but I don't know if that does
anything at all.
I doubt that the "-o nobarrier" is helping you much.
We run Centos 6.x in Lustre servers, where Lustre has been installed
from rpm's from Whamcloud/Intel build bot, and Ubuntu 10.04 in clients
with hand compiled kernel and Lustre. One MGC/MGS with twelve 15k-RPM
SAS disks in RAID-10 as MDT that is all but empty, and six variously
build RAID-6's in SAS-attached shelves in three OSS's.
ATdhvaannkcse for any help,
--
Olli Lounela
IT specialist and administrator
DNA sequencing and genomics
Institute of Biotechnology
University of Helsinki
_______________________________________________
Lustre-discuss mailing list
Lustre-discuss(a)lists.lustre.org
http://lists.lustre.org/mailman/listinfo/lustre-discuss
Cheers, Andreas
--
Andreas Dilger
Lustre Software Architect
Intel High Performance Data Division