6:46 a.m.
All,
we have got a test file system which had been created with Lustre 1.8
(or even 1.6), then briefly updated to 2.3, 2.4.1 and now to 2.4.2. On
this file system we now have a few directories that are inaccessible
after the latest upgrade. I believe they were accessible when we were
still running 2.4.1 but I'm not sure.
All clients are currently running 1.8.9.
Trying to ls one of the directories does generate an error on the
command line, but nothing in any of the system logs that I could find.
[bnh65367@p60-storage ~]$ ls -l /mnt/play01 |grep p60
ls: cannot access /mnt/play01/p45: No such file or directory
ls: cannot access /mnt/play01/p60: No such file or directory
d?????????? ? ? ? ? ? p60
[bnh65367@p60-storage ~]$ ls -l /mnt/play01/p60
ls: cannot access /mnt/play01/p60: No such file or directory
[bnh65367@p60-storage ~]$
Trying to touch one of the missing directories results in this on the
MDS and an input output error on the client command line.
Feb 11 19:13:23 cs04r-sc-mds02-03 kernel: LustreError:
14367:0:(mdt_open.c:1694:mdt_reint_open()) play01-MDT0000: name p60
present, but fid [0x45828f:0x7f3b41ef:0x0] invalid
I'm currently trying to understand if this is something that is
expected? Something we're likely to see if we upgrade directly from 1.8
to 2.4.2 on our production file systems? And of course we need to fix
it. To me it looks like LU-3934 could be related, though if I understand
that bug correctly, it should be fixed? Maybe it'll fix itself (by
automatically starting OI scrub?)?
Is this sufficiently different from LU-3934 and unexpected that I should
open a new ticket?
The file system has been upgrade a few hours ago, lctl get_param
'osd-ldiskfs.\*.oi_scrub on the MDS reports the status init for both MDT
and MGT (see below), does this mean it hasn't been started and I should
start it? How would I start it?
sudo lctl get_param 'osd-ldiskfs.\*.oi_scrub'
osd-ldiskfs.MGS.oi_scrub=
name: OI_scrub
magic: 0x4c5fd252
oi_files: 64
status: init
flags:
param:
time_since_last_completed: N/A
time_since_latest_start: N/A
time_since_last_checkpoint: N/A
latest_start_position: N/A
last_checkpoint_position: N/A
first_failure_position: N/A
checked: 0
updated: 0
failed: 0
prior_updated: 0
noscrub: 0
igif: 0
success_count: 0
run_time: 0 seconds
average_speed: 0 objects/sec
real-time_speed: N/A
current_position: N/A
osd-ldiskfs.play01-MDT0000.oi_scrub=
name: OI_scrub
magic: 0x4c5fd252
oi_files: 64
status: init
flags:
param:
time_since_last_completed: N/A
time_since_latest_start: N/A
time_since_last_checkpoint: N/A
latest_start_position: N/A
last_checkpoint_position: N/A
first_failure_position: N/A
checked: 0
updated: 0
failed: 0
prior_updated: 0
noscrub: 0
igif: 0
success_count: 0
run_time: 0 seconds
average_speed: 0 objects/sec
real-time_speed: N/A
current_position: N/A
[bnh65367@cs04r-sc-mds02-03 ~]$
I've also so far failed to find any good documentation on OI scrub, any
pointers will be welcome.
Kind regards,
Frederik
--
Frederik Ferner
Senior Computer Systems Administrator phone: +44 1235 77 8624
Diamond Light Source Ltd. mob: +44 7917 08 5110
(Apologies in advance for the lines below. Some bits are a legal
requirement and I have no control over them.)
--
This e-mail and any attachments may contain confidential, copyright and or privileged
material, and are for the use of the intended addressee only. If you are not the intended
addressee or an authorised recipient of the addressee please notify us of receipt by
returning the e-mail and do not use, copy, retain, distribute or disclose the information
in or attached to the e-mail.
Any opinions expressed within this e-mail are those of the individual and not necessarily
of Diamond Light Source Ltd.
Diamond Light Source Ltd. cannot guarantee that this e-mail or any attachments are free
from viruses and we cannot accept liability for any damage which you may sustain as a
result of software viruses which may be transmitted in or with the message.
Diamond Light Source Limited (company no. 4375679). Registered in England and Wales with
its registered office at Diamond House, Harwell Science and Innovation Campus, Didcot,
Oxfordshire, OX11 0DE, United Kingdom