I have 1 MDS, 2 OSS with 6 OSTs each and 4 Lustre Client. I have Hadoop
setup with 1 Namenode and 3 datanode running on lustre clients. While I
have 18GB input data in the form of many small files for wordcount, it took
1 hour time (striping: -1).
I have been running this on Ethernet. I have ordered for Infiniband QDR
which will reach in 2 weeks time.
Anyone who tried it with Infiniband. What time it get reduce if I use
infiniband switch and cable instead of Ethernet. Roughly how much lesser
time and efficient it shows with Infiniband. Please do share your
experience if you have used infiniband.
Show replies by thread