3:30 p.m.
I ran hadoop over lustre with 1 Namenode and 3 datanode running on Lustre
client. Here is my findings:
Scenerio1: 1 MDS, 2 OSS/OST, 3 Lustre Clients (1 NameNode and 2 DataNode),
Stripping : -1, Dataset: 18GB, Reducer: 20
Time taken: 59 min. 52 sec
Scenerio2: 1 MDS, 2 OSS/OST, 3 Lustre Clients (1 NameNode and 2 DataNode),
Stripping : -1, Dataset: 18GB, Reducer:30
Time Taken: 1 Hour . 5min
Ques: Did the time interval increase due to increase in reducer?
I have been using Ethernet. How much time(guess) will it take if I go for
Infiniband?
6:31 p.m.
On 2013/08/04 11:30 PM, "linux freaker" <linuxfreaker(a)gmail.com> wrote:
I ran hadoop over lustre with 1 Namenode and 3 datanode running on
Lustre
client. Here is my findings:
Scenerio1: 1 MDS, 2 OSS/OST, 3 Lustre Clients (1 NameNode and 2
DataNode), Stripping : -1, Dataset: 18GB, Reducer: 20
Time taken: 59 min. 52 sec
Scenerio2: 1 MDS, 2 OSS/OST, 3 Lustre Clients (1 NameNode and 2
DataNode), Stripping
: -1, Dataset: 18GB, Reducer:30
Time Taken: 1 Hour . 5min
Ques: Did the time interval increase due to increase in reducer?
Since you have parallelism in the clients, you probably shouldn't be using
striping = -1, but the default striping = 1.
I have been using Ethernet. How much time(guess) will it take if I go
for
Infiniband?
This is totally workload dependent. For Lustre metadata operations, IB
can be 5-10x faster, less so for IO operations. We'd be interested to
hear what kind of improvement you get for your Hadoop workload.
Cheers, Andreas
--
Andreas Dilger
Lustre Software Architect
Intel High Performance Data Division
4:32 a.m.
New subject: Upgrading 1.8.X -> 2.X
Lustre fans,
We have four lustre filesystems in production, and are considering running Lustre 2.X in
the next year. I'd like a bit of advice on upgrade paths, and I apologize if some of
this has been covered before - some of the details are eluding me. The main question is,
can the MGS be upgraded to 2.X on its own? If so, would it still support Lustre 1.8.3 OSSs
and MDSs? Would a Lustre 2.X MGS be possible if all the other systems were at Lustre 1.8.7
or better?
Current setup:
MGS - Lustre 1.8.3
2 filesystems (MDS and OSSs) - Lustre 1.8.3
2 filesystems (MDS and OSSs) - Lustre 1.8.7
No automatic failover configured
Clients - 1.8.7
Simple but very intrusive upgrade (long downtime):
Take everything offline and upgrade it all to 2.X at once
Not at all intrusive (only downtime for MGS):
Upgrade MGS to 2.X
Upgrade everything else as convenient (probably over the course of a year)
Somewhat intrusive (downtime for 2 filesystems):
Upgrade oldest filesystems to Lustre 1.8.7
Upgrade MGS to 2.X
Upgrade everything else as convenient
We would like to test Lustre 2.X with our current clients (with new servers and storage),
and only option 2 could be done in the short term, but we want to minimize risk to our
existing filesystems.
Thanks for any advice you can offer,
John Richards
john.richards(a)icecube.wisc.edu
2:15 a.m.
New subject: Upgrading 1.8.X -> 2.X
On 2013-04-26, at 12:32, "John Richards" <john.richards(a)icecube.wisc.edu>
wrote:
Lustre fans,
We have four lustre filesystems in production, and are considering running Lustre 2.X in
the next year. I'd like a bit of advice on upgrade paths, and I apologize if some of
this has been covered before - some of the details are eluding me. The main question is,
can the MGS be upgraded to 2.X on its own? If so, would it still support Lustre 1.8.3 OSSs
and MDSs? Would a Lustre 2.X MGS be possible if all the other systems were at Lustre 1.8.7
or better?
It isn't possible to run different 1.8/2.x versions on the servers, and this includes
the MGS. As it stands in 1.8 and 2.x MGS code it knows far too much about the internal
details of how the servers are configured, yet the way this happens is somewhat specific
to each version of Lustre. In 2.4 this has been improved somewhat, but I think some more
separation is still needed.
Probably the way you want to move forward is to split your single MGS into per-filesystem
MGS (possibly running on the backup MDS), and then they can be upgraded together with
their respective filesystems.
Cheers, Andreas
Current setup:
MGS - Lustre 1.8.3
2 filesystems (MDS and OSSs) - Lustre 1.8.3
2 filesystems (MDS and OSSs) - Lustre 1.8.7
No automatic failover configured
Clients - 1.8.7
Simple but very intrusive upgrade (long downtime):
Take everything offline and upgrade it all to 2.X at once
Not at all intrusive (only downtime for MGS):
Upgrade MGS to 2.X
Upgrade everything else as convenient (probably over the course of a year)
Somewhat intrusive (downtime for 2 filesystems):
Upgrade oldest filesystems to Lustre 1.8.7
Upgrade MGS to 2.X
Upgrade everything else as convenient
We would like to test Lustre 2.X with our current clients (with new servers and storage),
and only option 2 could be done in the short term, but we want to minimize risk to our
existing filesystems.
Thanks for any advice you can offer,
John Richards
john.richards(a)icecube.wisc.edu
_______________________________________________
HPDD-discuss mailing list
HPDD-discuss(a)lists.01.org
https://lists.01.org/mailman/listinfo/hpdd-discuss
5:39 a.m.
New subject: Upgrading 1.8.X -> 2.X
On Apr 29, 2013, at 11:15 , "Dilger, Andreas" <andreas.dilger(a)intel.com>
wrote:
On 2013-04-26, at 12:32, "John Richards"
<john.richards(a)icecube.wisc.edu> wrote:
> Lustre fans,
>
> We have four lustre filesystems in production, and are considering running Lustre 2.X
in the next year. I'd like a bit of advice on upgrade paths, and I apologize if some
of this has been covered before - some of the details are eluding me. The main question
is, can the MGS be upgraded to 2.X on its own? If so, would it still support Lustre 1.8.3
OSSs and MDSs? Would a Lustre 2.X MGS be possible if all the other systems were at Lustre
1.8.7 or better?
It isn't possible to run different 1.8/2.x versions on the servers, and this includes
the MGS. As it stands in 1.8 and 2.x MGS code it knows far too much about the internal
details of how the servers are configured, yet the way this happens is somewhat specific
to each version of Lustre. In 2.4 this has been improved somewhat, but I think some more
separation is still needed.
Probably the way you want to move forward is to split your single MGS into per-filesystem
MGS (possibly running on the backup MDS), and then they can be upgraded together with
their respective filesystems.
Cheers, Andreas
Andreas,
Thanks for the suggestion - per filesystem MGSs would allow us to upgrade individual
filesystems without tackling them all at once.
Clients only maintain communication with a single MGS, correct? We usually mount multiple
Lustre filesystems on clients at the same time, and we'd have to do some juggling to
handle that. I remember someone saying you could mount filesystems from different MGSs on
the same client, but would have to expect that the client would only receive updates from
the last MGS contacted. Or that could be an unstable situation (one that we should avoid
entirely) and we'd need to restrict each client to a single filesystem (and MGS) at a
time.
Even so, it is nice to have more options. I appreciate the help.
John
john.richards(a)icecube.wisc.edu
3429
days inactive
3449
days old
4 comments
3 participants
participants (3)
-
Dilger, Andreas -
John Richards -
linux freaker