Dan Huang
Jun Wang
ABSTRACT
Today, HPC clusters commonly use Resource Management Systems such as PBS and TORQUE to share physical resources. These systems enable resources to be shared by assigning nodes to users exclusively in non-overlapping time slots. With virtualization technology, users can run their applications on the same node with low mutual interference. However, the overhead introduced by the virtual machine monitor or hypervisor is too high to be accepted, because efficiency is key to many HPC applications. OS-level virtualization (such as Linux Containers) offers a lightweight virtualization layer, which promises a near-native performance and is adopted by some BigData resource sharing platforms such as Mesos. Nevertheless, OS-level virtualization's overhead and isolation on block devices have not been completely evaluated, especially when applied to a shared distributed/parallel file system (D/PFS) such as HDFS or Lustre. In this paper, we thoroughly evaluate the overhead and isolation involved in sharing block I/O via OS-level virtualization on the local disk and D/PFSs. Meanwhile, to assign D/PFS storage resources to users, a middleware system is proposed and implemented to bridge the configuration gap between virtual clusters and remote D/PFSs.
- Docker, https://www.docker.com/.Google Scholar
- KVM. http://www.linux-kvm.org/page/Main_Page.Google Scholar
- Linux container, https://linuxcontainers.org/.Google Scholar
- Lustre filesystem. http://www.lustre.org/.Google Scholar
- MPI-IO Test. http://institute.lanl.gov/data/software/.Google Scholar
- Openvz, http://www.openvz.org.Google Scholar
- Paraview, http://www.paraview.org/.Google Scholar
- TORQUE Resource Manager http://www.adaptivecomputing.com/products/open-source/torque/.Google Scholar
- Vmware, http://www.vmware.com/.Google Scholar
- Xen, http://www.xenproject.org.Google Scholar
- Dhruba Borthaku. The Hadoop Distributed File System: Architecture and Design.Google Scholar
- J. Dean and S. Ghemawat. Mapreduce: Simplified data processing on large clusters. In USENIX Symposium on Operating Systems Design and Implementation (OSDI), 2004. Google ScholarDigital Library
- Garth Gibson, Gary Grider, Andree Jacobson, and Wyatt Lloyd. Probe: A thousand-node experimental cluster for computer systems research. USENIX; login, 38(3), 2013.Google Scholar
- Benjamin Hindman, Andy Konwinski, Matei Zaharia, Ali Ghodsi, Anthony D. Joseph, Randy Katz, Scott Shenker, and Ion Stoica. Mesos: a platform for fine-grained resource sharing in the data center. In Proceedings of the 8th USENIX conference on Networked systems design and implementation, NSDI'11, pages 22--22, Berkeley, CA, USA, 2011. USENIX Association. Google ScholarDigital Library
- Nikolaus Huber, Marcel von Quast, Michael Hauck, and Samuel Kounev. Evaluating and modeling virtualization performance overhead for cloud environments. In CLOSER, pages 563--573, 2011.Google Scholar
- Junbin Kang, Benlong Zhang, Tianyu Wo, Chunming Hu, and Jinpeng Huai. Multilanes: providing virtualized storage for os-level virtualization on many cores. In FAST, pages 317--329, 2014. Google ScholarDigital Library
- Mingliang Liu, Ye Jin, Jidong Zhai, Yan Zhai, Qianqian Shi, Xiaosong Ma, and Wenguang Chen. Acic: automatic cloud i/o configurator for hpc applications. In High Performance Computing, Networking, Storage and Analysis (SC), 2013 International Conference for, pages 1--12. IEEE, 2013. Google ScholarDigital Library
- Shuangcheng Niu, Jidong Zhai, Xiaosong Ma, Xiongchao Tang, and Wenguang Chen. Cost-effective cloud hpc resource provisioning by building semi-elastic virtual clusters. In High Performance Computing, Networking, Storage and Analysis (SC), 2013 International Conference for, pages 1--12. IEEE, 2013. Google ScholarDigital Library
- Jongse Park, Daewoo Lee, Bokyeong Kim, Jaehyuk Huh, and Seungryoul Maeng. Locality-aware dynamic vm reconfiguration on mapreduce clouds. In Proceedings of the 21st international symposium on High-Performance Parallel and Distributed Computing, pages 27--36. ACM, 2012. Google ScholarDigital Library
- Stephen Soltesz, Herbert Pötzl, Marc E Fiuczynski, Andy Bavier, and Larry Peterson. Container-based operating system virtualization: a scalable, high-performance alternative to hypervisors. In ACM SIGOPS Operating Systems Review, volume 41, pages 275--287. ACM, 2007. Google ScholarDigital Library
- Tom White. Hadoop: The Definitive Guide. O'Reilly Media, original edition, June 2009, ISBN: 0596521979. Google ScholarDigital Library
- Miguel G Xavier, Marcelo Veiga Neves, Fabio D Rossi, Tiago C Ferreto, Timoteo Lange, and Cesar AF De Rose. Performance evaluation of container-based virtualization for high performance computing environments. In Parallel, Distributed and Network-Based Processing (PDP), 2013 21st Euromicro International Conference on, pages 233--240. IEEE, 2013. Google ScholarDigital Library
Recommendations
Optimizing virtual machines using hybrid virtualization
Minimizing virtualization overhead and improving the reliability of virtual machines are challenging when establishing virtual machine cluster. Paravirtualization and hardware-assisted virtualization are two mainstream solutions for modern system ...
System-Level Virtualization for High Performance Computing
PDP '08: Proceedings of the 16th Euromicro Conference on Parallel, Distributed and Network-Based Processing (PDP 2008)System-level virtualization has been a research topic since the 70's but regained popularity during the past few years because of the availability of efficient solution such as Xen and the implementation of hardware support in commodity processors (e.g. ...
Comments