Re: [HPDD-discuss] Lustre 2.5.1. HSM

Hi Aamir, Let me see if I can address some of your question inline. Malcolm. The coordinator acts like a job scheduler, queuing and dispatching requests to HSM agents. Each request is sent to one of the available agents and the requests work at the granularity of an individual file. This means that individual files are _not_ split up and sent in parallel across multiple agents. That is correct. Each agent runs one copytool, so if multiple copytools are required, they must each run on their own machine. Yes, if you require an unlimited number of copytools, one must match those instances with an unlimited supply of servers. The software does not impose a limit on the number of instances but other considerations may force a practical limit :). No. Generally, a configuration management system (CF-Engine, Puppet, Chef) could be employed to monitor and restart services if they fail. Intel Manager for Lustre also monitors copytool instances and restarts them on fail. The Coordinator will load balance requests across the copytool instances. Lustre servers can be integrated into an HA cluster framework such as Pacemaker + Corosync. The coordinator runs as a service thread on the metadata server and so will failover if/when the MDT fails over. To provide HA in the HSM Agents, install multiple agent servers, each with their own instance. Yes. Just launch multiple copytools, one per server. Each instance will automatically register with the coordinator. The coordinator will automatically schedule work to the pool of registered instances, similar to how a job scheduler operates. I think that's the latest publically available documentation.