Re: [HPDD-discuss] Removal of SOM code

Oh no! I had just been talking about using this feature :( Even in it's current form which I agree isn't ideal, I think it could be helpful for a project like fssstats. Fsstats was an open effort to gather and collect filesystems data (see http://www.pdsi-scidac.org/fsstats/). It has unfortunately somewhat died off due to loss of funding, but I think was a good idea that helped the community, especially researchers. I'd really like to try and revive the open data initiative, and saw som as a possible avenue to start collecting the data. I don't think statahead will provide the information needed but haven't had a chance to look at it. Thoughts? On Fri, Feb 6, 2015 at 3:33 PM, Dilger, Andreas <andreas.dilger(a)intel.com&gt; wrote: > The Size on MDT (SOM) feature has been in a prototype state for several > years, with no signs of moving beyond this prototype stage. > > Several problems exist in the code today, primarily that recovery is not > really implemented, yet the existing code adds complexity on the clients > and servers. Without proper recovery, the current code risks file data > loss if the SOM data isn't updated on the MDS consistently with data > writes to the OST. > > > We're planning to remove the SOM code from the master branch as a result, > tracked under https://urldefense.proofpoint.com/v2/url?u=https-3A__jira.hpdd.intel.com_... > - https://urldefense.proofpoint.com/v2/url?u=http-3A__review.whamcloud.com_... > - https://urldefense.proofpoint.com/v2/url?u=http-3A__review.whamcloud.com_... > - https://urldefense.proofpoint.com/v2/url?u=http-3A__review.whamcloud.com_... > - https://urldefense.proofpoint.com/v2/url?u=http-3A__review.whamcloud.com_... > > Some of the performance improvements of SOM have been implemented by > statahead. > > I think a case could be made for a very stripped down SOM to be > implemented in the future, that only deals with single-client writers and > synchronously invalidates the file size on open-for-write, which isn't so > bad with flash storage for the MDT as is typical today. The size of files > that do not get set at initial write or are invalidated by an open can be > updated asynchronously by LFSCK doing a periodic scan in the background. > Since this stripped-down implementation would have very little to do with > the current implementation, there isn't much benefit to even trying to fix > the current code in place. > > I definitely prefer presenting about new features going into Lustre, but I > also think it is important that people are aware when a semi-feature like > this is being removed. I don't believe that anyone is actually using this > feature today, and the reduction in code maintenance and complexity will > help both ongoing maintenance and bug fixing, as well as make it a that > much easier for new developers to understand the code. > > Cheers, Andreas > -- > Andreas Dilger > > Lustre Software Architect > Intel High Performance Data Division > > > _______________________________________________ > HPDD-discuss mailing list > HPDD-discuss(a)lists.01.org > https://urldefense.proofpoint.com/v2/url?u=https-3A__lists.01.org_mailman... -- Meghan McClelland · Senior Product Manager Seagate Technology, LLC mobile: +1 (505) 695 0065 www.seagate.com _______________________________________________ HPDD-discuss mailing list HPDD-discuss(a)lists.01.org https://lists.01.org/mailman/listinfo/hpdd-discuss