Hi Andreas,

Thanks for the response to my queries. :)
I was under the assumption that directory data would be striped across OSTs just like regular files. Your answer clarified that and my assumption is proved wrong!!
Also, as an add-on question, could you please provide your opinion regarding directory creation vs file creation overhead in Lustre. (In normal Linux systems, directory creation is generally considered an expensive operation. So just wanted to clarify on that in terms of Lustre). 
The background is I'm trying to create a directory for every file and then write each write request (for that file) into a separate file under the newly created directory. (so number of file creates in actual Lustre = number of directories in my modified form and each write creates a new file). I know it's not the normal Lustre operation and there is an additional overhead to MDS in first creating the directory and then writing to the files under it instead of just creating a single file. Could you please help me in understanding if Directory creation / Directory lookup (for reads) would lead to lot of additional overhead compared to the present case.

Have a great weekend!! :)

Regards,
Akhilesh.


On Sat, Sep 5, 2015 at 7:57 PM, Dilger, Andreas <andreas.dilger@intel.com> wrote:
On Sep 5, 2015, at 14:37, Akhilesh Gadde <akhilesh.gadde@stonybrook.edu> wrote:
> I have a query regarding the file metadata stored in MDS/MDT.
>
> I am trying to understand what information would be normally present in Inode EA attributes for a directory (not file). I tried searching for it but could not find much information.
>
> - When a new file is created, MDS would create a new FID with EA attributes. File EA given to client would contain objects on which the data would be written and their corresponding OSTs.

This is the "lov" xattr that holds the file layout. The Lustre FID is also stored in the "lma" xattr on the inode (both file and directory inodes).

Lustre also stores the "link" xattr the holds the filename and parent FID for each link to the file. That allows generating a pathname from the FID for each file.

> (This ensures separation of metadata from actual data since now the client can write directly to OSTs without contacting MDS)

Correct.

> - I would like to know what happens similarly when a new directory is created. Does MDS also allocate objects and their corresponding OSTs (considering the directory data is nothing but it's containing filenames and their inode numbers)

No, the directory is held only on the MDT.  It also has an "lma" xattr to hold the FID. In Lustre 2.8 and later a single directory may be striped across multiple MDTs, analogous to how a file is striped over multiple OSTs, in which case it will have an "lov" xattr to hold the directory striping layout.

> Could you please help me in clarifying the above query and also correct me if I'm misinterpreting some information.
>
> Regards,
> Akhilesh.
> _______________________________________________
> HPDD-discuss mailing list
> HPDD-discuss@lists.01.org
> https://lists.01.org/mailman/listinfo/hpdd-discuss