Re: [HPDD-discuss] Lustre networking issues, multi-homed servers

Hello, I just upgraded a Lustre 1.8.7 installation to version 2.5.3. The Lustre servers are connected via IB and Ethernet, some of the clients have both networks, and some of the clients have Ethernet only. I'm having a problem where the Ethernet-only clients appear to be attempting to contact the servers via their IB addresses, and are failing to do so. As far as I can tell the NIDS are correct on servers and clients, so I'm not sure where things are going wrong. The Ethernet networks are 10.100.* and the IB network is 192.168.* below. MDS/MGS: # lctl list_nids 192.168.129.250@o2ib 10.100.129.250@tcp OSSes: # lctl list_nids 192.168.129.252@o2ib 10.100.129.252@tcp # lctl list_nids 192.168.129.249@o2ib 10.100.129.249@tcp # lctl list_nids 192.168.129.251@o2ib 10.100.129.251@tcp Client: # lctl list_nids 10.100.135.131@tcp On the client: # mount -t lustre 10.100.129.250@tcp:/lustre_1 /lustre # df <HANGS> Jan 22 10:49:01 compute-f09-1 kernel: Lustre: Lustre: Build Version: 2.5.3-RC1--PRISTINE-2.6.32-504.3.3.el6.x86_64 Jan 22 10:49:01 compute-f09-1 kernel: Lustre: client wants to enable acl, but mdt not! Jan 22 10:49:01 compute-f09-1 kernel: Lustre: Layout lock feature supported. Jan 22 10:49:01 compute-f09-1 kernel: Lustre: Mounted lustre_1-client Jan 22 10:49:06 compute-f09-1 kernel: Lustre: 5545:0:(client.c:1918:ptlrpc_expire_one_request()) @@@ Request sent has timed out for sent delay: [sent 1421941741/real 0] req@ffff8808042bac00 x1491013983010920/t0(0) o8->lustre_1-OST000a-osc-ffff880823397400@192.168.129.249@tcp:28/4 lens 400/544 e 0 to 1 dl 1421941746 ref 2 fl Rpc:XN/0/ffffffff rc 0/-1 Jan 22 10:49:14 compute-f09-1 kernel: LustreError: 5630:0:(llite_lib.c:1624:ll_statfs_internal()) obd_statfs fails: rc = -5 For some reason the client appears to be trying to connect to the 192.168 (IB) address, even though it's not one of its networks. Can someone please shed some light as to what I'm missing? Thanks, Kevin --- Kevin Hildebrand University of Maryland Division of IT _______________________________________________ HPDD-discuss mailing list HPDD-discuss(a)lists.01.org https://lists.01.org/mailman/listinfo/hpdd-discuss