TITLE

BUILDING A PRIVATE HPC CLOUD FOR COMPUTE AND DATA-INTENSIVE APPLICATIONS

AUTHOR(S)
Taifi, Moussa; Khreishah, Abdallah; Shi, Justin Y.
PUB. DATE
April 2013
SOURCE
International Journal on Cloud Computing: Services & Architectur;Apr2013, Vol. 3 Issue 2, p1
SOURCE TYPE
Academic Journal
DOC. TYPE
Article
ABSTRACT
Traditional HPC (High Performance Computing) clusters are best suited for well-formed calculations. The orderly batch-oriented HPC cluster offers maximal potential for performance per application, but limits resource efficiency and user flexibility. An HPC cloud can host multiple virtual HPC clusters, giving the scientists unprecedented flexibility for research and development. With the proper incentive model, resource efficiency will be automatically maximized. In this context, there are three new challenges. The first is the virtualization overheads. The second is the administrative complexity for scientists to manage the virtual clusters. The third is the programming model. The existing HPC programming models were designed for dedicated homogeneous parallel processors. The HPC cloud is typically heterogeneous and shared. This paper reports on the practice and experiences in building a private HPC cloud using a subset of a traditional HPC cluster. We report our evaluation criteria using Open Source software, and performance studies for compute-intensive and data-intensive applications. We also report the design and implementation of a Puppet-based virtual cluster administration tool called HPCFY. In addition, we show that even if the overhead of virtualization is present, efficient scalability for virtual clusters can be achieved by understanding the effects of virtualization overheads on various types of HPC and Big Data workloads. We aim at providing a detailed experience report to the HPC community, to ease the process of building a private HPC cloud using Open Source software.
ACCESSION #
88067547

 

Related Articles

  • Events Diary.  // Scientific Computing World;Feb/Mar2014, Issue 134, p35 

    A calendar of world events in February-September 2014 is presented which includes Health Informatics Congress, 11th Annual 2014 HPC Linux for Wall Street, and Big Data TechCon 2014.

  • Events Diary.  // Scientific Computing World;Oct/Nov2014, Issue 138, p54 

    A calendar of events related to the high performance computing industry is presented including the International Conference on Big Data Science and Computing, the Lab Innovations 2014, and the 25th Anniversary Machine Evaluation Workshop.

  • Advanced Computing.  // American Ceramic Society Bulletin;Jan2007, Vol. 86 Issue 1, p7 

    The article reports on the move of Lawrence Berkeley National Laboratories to lead the five of the 30 projects under the second round of the Scientific Discovery of the Computing (SciDAC) program. The SciDAC 2 projects include computational science projects which aimed to accelerate research in...

  • Big Data: It's not Just Big. Skinner, David // Scientific Computing;Aug2013 Supplement, p18 

    The article discusses the importance of data in computing. It mentions that data from simulation and experiments provide the basis for advanced scientific research and knowledge discovery. It further states that high performance computing (HPC) enables high speed large-scale simulation and data...

  • Accelerating Data-intensive Computing. Tracy, Suzanne // Scientific Computing;Apr2013 Supplement, p4 

    An introduction is presented in which the editor discusses various reports within the issue on topics including the analysis and management of big data, deployment of the Hadoop framework, and the role of high performance computing (HPC) in bioscience.

  • Events Diary.  // Scientific Computing World;Dec2014/Jan2015, Issue 139, p27 

    A calendar of events for the international high-performance computing (HPC) sector from February 7 - October 1, 2015 is presented which includes the Advances and Progress in Drug Design, High Performance Computing and Big Data 2015 and Americas Altair Technology Conference 2015.

  • FastaValidator: an open-source Java library to parse and validate FASTA formatted sequences. Waldmann, Jost; Gerken, Jan; Hankeln, Wolfgang; Schweer, Timmy; Gl√∂ckner, Frank Oliver // BMC Research Notes;2014, Vol. 7 Issue 1, p2 

    Background Advances in sequencing technologies challenge the efficient importing and validation of FASTA formatted sequence data which is still a prerequisite for most bioinformatic tools and pipelines. Comparative analysis of commonly used Bio*-frameworks (BioPerl, BioJava and Biopython) shows...

  • Feuding Science.  // Odyssey;Jan2002, Vol. 11 Issue 1, p48 

    Focuses on the rivalry in science discoveries. Conflict between inventors Thomas Edison and Nokola Tesla on the electrical current; Advancement of science for human; Role of scientists in the industry.

  • More than one Amazon at the forefront of innovation. Murray, Lain // Marketing Week;9/4/2003, Vol. 26 Issue 36, p94 

    Focuses on the scientific discoveries of the scientists in Brazil. Treatment of burns using frog skin; Invention of robotic tongue by Antonio Riul; Facilitation of laughter therapy.

Share

Read the Article

Courtesy of VIRGINIA BEACH PUBLIC LIBRARY AND SYSTEM

Sorry, but this item is not currently available from your library.

Try another library?
Sign out of this library

Other Topics