[HPDD-discuss] Find run-away file

Hello, Lately, one of our (12TB) OST jumped from ~50% to 100% capacity in a matter of hours. We switched that OST to INACTIVE before it reached 100% but it kept filling, indicating an ongoing file write. At the time it reached 100%, we got an ENOSPC on one of our client: helvetix05: 2014-03-11 09:35:06 helvetix05 kernel: [1610930.474849] LustreError: 4431:0:(vvp_io.c:1022:vvp_io_commit_write()) Write page 1572607068 of inode ffff8800c96277c8 failed -28 helvetix05: 2014-03-11 09:35:06 helvetix05 kernel: [1610930.692143] LustreError: 4431:0:(vvp_io.c:1022:vvp_io_commit_write()) Write page 1572607068 of inode ffff8800c96277c8 failed -28 We tried to catch the run-away file using 'lfs find' but with a 250mio-files filesystem, this is no easy feat. We also asked the suspected user, but he has no idea what/how things went wrong. QUESTION: Can we assume the 1572607068 page figure point to a 6TB file (1572607068*4096 bytes) ? (this would be consistent with the given OST capacity figures) Is there a way to find which file corresponds to the ffff8800c96277c8 inode ? Is there a way to perform the equivalent of the 'lfs find' directly on the MDS (e.g. by mounting the underlying ldiskfs) ? Thanks for your help, Cédric -- Cédric Dufour @ Idiap Research Institute EPFL Engineer E-mail: mailto:cedric.dufour@idiap.ch Phone: +41 27 721 77 40 Fax: +41 27 721 77 12 Mail: Idiap Research Institute Case postale 592 Centre du Parc - Rue Marconi 19 1920 Martigny (VS) Suisse (Switzerland) Website: http://www.idiap.ch / http://www.idiap.ch/~cdufour <http://www.idiap.ch/%7Ecdufour>