7:37 p.m.
Hi everyone,
I installed the latest 2.5.3 version on the latest CentOS 6.5 version with
all upgrades and used the following commands to create a lustre file system:
1. Created /etc/modprobe.d/lustre.conf
options lnet networks=tcp0(eth0)
2. Started lnet
service lnet start
lctl net up
3. Created MGS and MDT:
mkfs.lustre --mgs /dev/sdb
mkfs.lustre --mdt --mgsnode=1.1.1.125@tcp0 --network=tcp0 --fsname=fs01
--index=0 /dev/sdc
4. Updated /etc/ldev.conf
mgs - MGS /dev/sdb -
mgs - fs01:MDT0000 /dev/sdc -
5. Started Lustre:
service lustre start MGS
service lustre start fs01:MDT0000
6. And getting the following error:
Sep 18 12:29:03 localhost kernel: LustreError: 11-0:
fs01-MDT0000-lwp-MDT0000: Communicating with 0@lo, operation mds_connect
failed with -11.
I checked that lctl ping 0@lo is working properly.
8:27 p.m.
Hello!
On Sep 18, 2014, at 5:37 AM, Dennis Zheleznyak wrote:
3. Created MGS and MDT:
mkfs.lustre --mgs /dev/sdb
mkfs.lustre --mdt --mgsnode=1.1.1.125@tcp0 --network=tcp0 --fsname=fs01 --index=0
/dev/sdc
You also need at least one (but possibly more) OSTs - the place where the actual data is
stored.
4. Updated /etc/ldev.conf
mgs - MGS /dev/sdb -
mgs - fs01:MDT0000 /dev/sdc -
5. Started Lustre:
service lustre start MGS
service lustre start fs01:MDT0000
6. And getting the following error:
Sep 18 12:29:03 localhost kernel: LustreError: 11-0: fs01-MDT0000-lwp-MDT0000:
Communicating with 0@lo, operation mds_connect failed with -11.
Yup, that's what you get if you don't have any OSTs.
Bye,
Oleg
11:46 p.m.
Hey Oleg,
Thanks for your quick response !
I created MGS, MDT and two OSTs before I started lustre service on both
computer - is it possible that the error is shown only the first time
starting the service ?
On Thu, Sep 18, 2014 at 1:27 PM, Drokin, Oleg <oleg.drokin(a)intel.com> wrote:
Hello!
On Sep 18, 2014, at 5:37 AM, Dennis Zheleznyak wrote:
> 3. Created MGS and MDT:
> mkfs.lustre --mgs /dev/sdb
> mkfs.lustre --mdt --mgsnode=1.1.1.125@tcp0 --network=tcp0 --fsname=fs01
--index=0 /dev/sdc
You also need at least one (but possibly more) OSTs - the place where the
actual data is stored.
> 4. Updated /etc/ldev.conf
> mgs - MGS /dev/sdb -
> mgs - fs01:MDT0000 /dev/sdc -
>
> 5. Started Lustre:
> service lustre start MGS
> service lustre start fs01:MDT0000
>
> 6. And getting the following error:
> Sep 18 12:29:03 localhost kernel: LustreError: 11-0:
fs01-MDT0000-lwp-MDT0000: Communicating with 0@lo, operation mds_connect
failed with -11.
Yup, that's what you get if you don't have any OSTs.
Bye,
Oleg
1:09 a.m.
On Sep 18, 2014, at 5:37 AM, Dennis Zheleznyak <dennis(a)eshkol.com.co>
wrote:
3. Created MGS and MDT:
mkfs.lustre --mgs /dev/sdb
mkfs.lustre --mdt --mgsnode=1.1.1.125@tcp0 --network=tcp0 --fsname=fs01 --index=0
/dev/sdc
Based on the error message, I am assuming the MGT and MDT are on the same node. If so,
you could try omitting the "--mgsnode" and "--network" options to see
if that is playing any role in the failure. Without those options, I believe the MDT will
default to contacting the localhost and maybe that will work.
I also came across this ticket: https://jira.hpdd.intel.com/browse/LU-4475.
When you try to mount the osts, do you see "Transport endpoint is not connected"
error messages?
--
Rick Mohr
Senior HPC System Administrator
National Institute for Computational Sciences
http://www.nics.tennessee.edu
1:56 a.m.
Hi Rick,
I tried your suggestion, however, --mgsnode is mandatory, otherwise it
won't mount after typing service lustre start
Thanks for your help, I believe Oleg's tip helped, adding OST helped
although the message still appears
On Thu, Sep 18, 2014 at 6:09 PM, Mohr Jr, Richard Frank (Rick Mohr) <
rmohr(a)utk.edu> wrote:
On Sep 18, 2014, at 5:37 AM, Dennis Zheleznyak <dennis(a)eshkol.com.co>
wrote:
> 3. Created MGS and MDT:
> mkfs.lustre --mgs /dev/sdb
> mkfs.lustre --mdt --mgsnode=1.1.1.125@tcp0 --network=tcp0 --fsname=fs01
--index=0 /dev/sdc
Based on the error message, I am assuming the MGT and MDT are on the same
node. If so, you could try omitting the "--mgsnode" and "--network"
options to see if that is playing any role in the failure. Without those
options, I believe the MDT will default to contacting the localhost and
maybe that will work.
I also came across this ticket: https://jira.hpdd.intel.com/browse/LU-4475
.
When you try to mount the osts, do you see "Transport endpoint is not
connected" error messages?
--
Rick Mohr
Senior HPC System Administrator
National Institute for Computational Sciences
http://www.nics.tennessee.edu
2:14 a.m.
On Sep 18, 2014, at 11:56 AM, Dennis Zheleznyak <dennis(a)eshkol.com.co>
wrote:
I tried your suggestion, however, --mgsnode is mandatory, otherwise
it won't mount after typing service lustre start
Sorry. I was thinking along the lines of a combined MGT/MDT (which is what I have) and
forgot the fact that your MGT and MDT are on different devices even though they are on the
same host.
Glad to hear Oleg's suggestion worked out for you.
--
Rick Mohr
Senior HPC System Administrator
National Institute for Computational Sciences
http://www.nics.tennessee.edu
11:43 a.m.
Basically MDS will not let clients in until it was able to connect to at least to one
ost.
So if you start your client at a time when MDS just started up and did not connct to an
OST yet, you'll get this error.
On Sep 18, 2014, at 9:46 AM, Dennis Zheleznyak wrote:
Hey Oleg,
Thanks for your quick response !
I created MGS, MDT and two OSTs before I started lustre service on both computer - is it
possible that the error is shown only the first time starting the service ?
On Thu, Sep 18, 2014 at 1:27 PM, Drokin, Oleg <oleg.drokin(a)intel.com> wrote:
Hello!
On Sep 18, 2014, at 5:37 AM, Dennis Zheleznyak wrote:
> 3. Created MGS and MDT:
> mkfs.lustre --mgs /dev/sdb
> mkfs.lustre --mdt --mgsnode=1.1.1.125@tcp0 --network=tcp0 --fsname=fs01 --index=0
/dev/sdc
You also need at least one (but possibly more) OSTs - the place where the actual data is
stored.
> 4. Updated /etc/ldev.conf
> mgs - MGS /dev/sdb -
> mgs - fs01:MDT0000 /dev/sdc -
>
> 5. Started Lustre:
> service lustre start MGS
> service lustre start fs01:MDT0000
>
> 6. And getting the following error:
> Sep 18 12:29:03 localhost kernel: LustreError: 11-0: fs01-MDT0000-lwp-MDT0000:
Communicating with 0@lo, operation mds_connect failed with -11.
Yup, that's what you get if you don't have any OSTs.
Bye,
Oleg
2921
days inactive
2922
days old
6 comments
3 participants
participants (3)
-
Dennis Zheleznyak -
Drokin, Oleg -
Mohr Jr, Richard Frank (Rick Mohr)