[HPDD-discuss] Long req_waittime

Dear All, I am trying figure out what is causing long req_waittime for our OST’s. This looks really bad. Any tips on digging into this is greatly appreciated. I am checking the health of OST and all look okay. I will collect RPC stats and see if I see something there. Here is a sample output from client req_waittime 239189 req_active 239189 ost_connect 1 ost_statfs 254 ldlm_cancel 13 obd_ping 238907 sample from one OST: snapshot_time 1418769865.299088 req_waittime 1142788387 req_qdepth 1142788387 req_active 1142788387 req_timeout 1142788387 reqbuf_avail 2389588796 ldlm_glimpse_enqueue 7125998 ldlm_extent_enqueue 86631679 ost_setattr 136096 ost_create 33258 ost_destroy 757369 ost_connect 13145 ost_disconnect 4302 obd_ping 1048086540 are these cumulative stats, don’t believe so??? Best Regards, Amit