SIGN-IN

Publication: Job Allocation Strategies for Multiple High Performance Computing Clusters

All || By Area || By Year

Title Job Allocation Strategies for Multiple High Performance Computing Clusters
Authors/Editors* J. Qin and M. Bauer
Where published* Cluster Computing
How published* Journal
Year* 2010
Volume
Number
Pages to appear
Publisher
Keywords resource management, job co-allocation, job scheduling, HPC clusters, performance evaluation
Link
Abstract
To more effectively use a network of high performance computing clusters, allocating multi-process jobs across multiple connected clusters becomes an attractive possibility. This allocation process entails dividing the processes of a job among several clusters, which we refer to as co-allocation. Co-allocation offers the possibility of more efficient use of computer resources, reduced turn-around time and computations using numbers of processes larger than processes on any single cluster. In order to realize these possibilities, effective co-allocation, ultimately, depends on the inter-cluster communication cost. In this paper, we introduce a scalable co-allocation strategy called the Maximum Bandwidth Adjacent cluster Set (MBAS) strategy. The strategy makes use of two thresholds to control allocation: one to control the limit on bandwidth on usable inter-cluster communication links and another to control how jobs are split. A simulator that can simulate the dynamic behavior of jobs running across multiple clusters was developed and used to examine the performance of the MBAS co-allocation strategy. Our results indicate that by adjusting the thresholds for link level control and chunk size control in splitting jobs, the MBAS co-allocation strategy can significantly improve both user satisfaction and system utilization.
Go to Distributed Systems
Back to page 28 of list