10:48 p.m.
Hello,
I am in verse to setup Hadoop over lustre(replacing HDFS).
I have 1 MDS, 2 OSS/OST and 2 Lustre Client.
My MDS shows:
[code]
[root@MDS ~]# lctl list_nids
10.84.214.185@tcp
[/code]
Lustre Client shows:
[code]
[root@lustreclient1 ~]# lfs df -h
UUID bytes Used Available Use% Mounted on
lustre-MDT0000_UUID 4.5G 274.3M 3.9G 6% /mnt/lustre[MDT:0]
lustre-OST0000_UUID 5.9G 276.1M 5.3G 5% /mnt/lustre[OST:0]
lustre-OST0001_UUID 5.9G 276.1M 5.3G 5% /mnt/lustre[OST:1]
lustre-OST0002_UUID 5.9G 276.1M 5.3G 5% /mnt/lustre[OST:2]
lustre-OST0003_UUID 5.9G 276.1M 5.3G 5% /mnt/lustre[OST:3]
lustre-OST0004_UUID 5.9G 276.1M 5.3G 5% /mnt/lustre[OST:4]
lustre-OST0005_UUID 5.9G 276.1M 5.3G 5% /mnt/lustre[OST:5]
lustre-OST0006_UUID 5.9G 276.1M 5.3G 5% /mnt/lustre[OST:6]
lustre-OST0007_UUID 5.9G 276.1M 5.3G 5% /mnt/lustre[OST:7]
lustre-OST0008_UUID 5.9G 276.1M 5.3G 5% /mnt/lustre[OST:8]
lustre-OST0009_UUID 5.9G 276.1M 5.3G 5% /mnt/lustre[OST:9]
lustre-OST000a_UUID 5.9G 276.1M 5.3G 5%
/mnt/lustre[OST:10]
lustre-OST000b_UUID 5.9G 276.1M 5.3G 5%
/mnt/lustre[OST:11]
filesystem summary: 70.9G 3.2G 64.0G 5% /mnt/lustre
[/code]
Now I installed Hadoop on two Lustre Client(untouching MDS and OSS).
My core-site.xml shows:
[code]
<property>
<name>fs.default.name</name>
<value>file:///mnt/lustre</value>
</property>
<property>
<name>mapred.system.dir</name>
<value>${fs.default.name}/hadoop_tmp/mapred/system</value>
<description>The shared directory where MapReduce stores control
files.
</description>
</property>
[/code]
My conf/masters shows
[code]
lustreclient1
[/code]
My conf/slaves shows:
[code]
lustreclient1
lustreclient2
I have no idea if I need any further configuration file changes.
Do I need just the above configuration.
What about hdfs-site.xml and mapred-site.xml?