Horacio Andrés Lagar-Cavilla
Mahadev Satyanarayanan
ABSTRACT
Virtual Machine (VM) fork is a new cloud computing abstraction that instantaneously clones a VM into multiple replicas running on different hosts. All replicas share the same initial state, matching the intuitive semantics of stateful worker creation. VM fork thus enables the straightforward creation and efficient deployment of many tasks demanding swift instantiation of stateful workers in a cloud environment, e.g. excess load handling, opportunistic job placement, or parallel computing. Lack of instantaneous stateful cloning forces users of cloud computing into ad hoc practices to manage application state and cycle provisioning. We present SnowFlock, our implementation of the VM fork abstraction. To evaluate SnowFlock, we focus on the demanding scenario of services requiring on-the-fly creation of hundreds of parallel workers in order to solve computationally-intensive queries in seconds. These services are prominent in fields such as bioinformatics, finance, and rendering. SnowFlock provides sub-second VM cloning, scales to hundreds of workers, consumes few cloud I/O resources, and has negligible runtime overhead.
- S. F. Altschul, T. L. Madden, A. A. Schaffer, J. Zhang, Z. Zhang, W. Miller, and D. J. Lipman. Gapped BLAST and PSI-BLAST: a new generation of protein database search programs. Nucleic Acids Res., 25:3389--3402, 1997.Google Scholar
Cross Ref
- A. A. Apodaka and L. Gritz. Advanced Render-Man: Creating CGI for Motion Pictures. Academic Press, 2000. Google ScholarDigital Library
- Aqsis: Open source 3D rendering solution adhering to the RenderMan standard. http://aqsis.org/.Google Scholar
- P. Barham, B. Dragovic, K. Fraser, S. Hand, T. Harris, A. Ho, R. Neugebauer, I. Pratt, and A.Warfield. Xen and the Art of Virtualization. In Proc. 17th Symposium on Operating Systems Principles (SOSP), Bolton Landing, NY, October 2003. Google ScholarDigital Library
- P. J. Braam. The lustre storage architecture, 2002. http://www.lustre.org/docs/lustre.pdf.Google Scholar
- J. S. Chase, D. E. Irwin, L. E. Grit, J. D. Moore, and S. E. Sprenkle. Dynamic Virtual Clusters in a Grid Site Manager. In Proc. 12th Symposium on High Performance Distributed Computing (HPDC), Washington, DC, 2003. Google ScholarDigital Library
- C. Clark, K. Fraser, S. Hand, J. Gorm Hansen, E. Jul, C. Limpach, I. Pratt, and A. Warfield. Live Migration of Virtual Machines. In Proc. 2nd Symposium on Networked Systems Design and Implementation (NSDI), Boston, MA, May 2005. Google ScholarDigital Library
- B. Cully, G. Lefebvre, D. Meyer, M. Feeley, N. Hutchinson, and A. Warfield. Remus: High Availability via Asynchronous Virtual Machine Replication. In Proc. 5th Symposium on Networked Systems Design and Implementation (NSDI), San Francisco, CA, April 2008. Google ScholarDigital Library
- J. Dean and S. Ghemawat. MapReduce: Simplified Data Processing on Large Clusters. In Proc. 6th Symposium on Operating System Design and Implementation (OSDI), San Francisco, CA, December 2004. Google ScholarDigital Library
- samba.org. distcc: a fast, free distributed C/C++ compiler. http://distcc.samba.org/.Google Scholar
- European Bioinformatics Institute. ClustalW2. http://www.ebi.ac.uk/Tools/clustalw2/index.html.Google Scholar
- Amazon.com. EC2: Amazon Elastic Compute Cloud. http://aws.amazon.com/ec2/.Google Scholar
- W. Emeneker and D. Stanzione. Dynamic Virtual Clustering. In Proc. Cluster, Austin, TX, September 2007. Google ScholarDigital Library
- I. Foster, T. Freeman, K. Keahey, D. Scheftner, B. Sotomayor, and X. Zhang. Virtual Clusters for Grid Communities. In Proc. Cluster Computing and the Grid, Singapore, May 2006. Google ScholarDigital Library
- W. Gentzsch. Sun Grid Engine: Towards Creating a Compute Power Grid. In Proc. 1st Symposium on Cluster Computing and the Grid, Brisbane, Australia, May 2001. Google ScholarDigital Library
- Apache.org. Hadoop. http://hadoop.apache.org/core/.Google Scholar
- M. Hibler, R. Ricci, L. Stoller, J. Duerig, S. Guruprasad, T. Stack, K. Webb, and J. Lepreau. Large-scale Virtualization in the Emulab Network Testbed. In Proc. USENIX Annual Technical Conference, Boston, MA, June 2008. Google ScholarDigital Library
- M. Hibler, L. Stoller, J. Lepreau, R. Ricci, and C. Barb. Fast, Scalable Disk Imaging with Frisbee. In Proc. USENIX Annual Technical Conference, San Antonio, TX, June 2003.Google Scholar
- W. Huang, Q. Gao, J. Liu, and D. K. Panda. High Performance Virtual Machine Migration with RDMA over Modern Interconnects. In Proc. Cluster, Austin, TX, September 2007. Google ScholarDigital Library
- M. Kozuch and M. Satyanarayanan. Internet Suspend/ Resume. In Proc. 4th Workshop on Mobile Computing Systems and Applications (WMCSA), Callicoon, NY, June 2002. Google ScholarDigital Library
- H. A. Lagar-Cavilla, N. Tolia, E. de Lara, M. Satyanarayanan, and D. O'Hallaron. Interactive Resource-Intensive Applications Made Easy. In Proc. 8rm International Middleware Conference, Newport Beach, CA, November 2007. Google ScholarDigital Library
- M. McNett, D. Gupta, A. Vahdat, and G. Voelker. Usher: An Extensible Framework for Managing Clusters of Virtual Machines. In Proc. 21st LISA, Dallas, TX, November 2007. Google ScholarDigital Library
- Argonne National Laboratory. MPICH2. http://www.mcs.anl.gov/research/projects/mpich2/.Google Scholar
- National Center for Biotechnology Information. BLAST: Basic Local Alignment and Search Tool. http://blast.ncbi.nlm.nih.gov/Blast.cgi.Google Scholar
- RenderMan. https://renderman.pixar.com/.Google Scholar
- Platform Computing. Platform EGO Home. http://my.platform.com/products/platform-ego-de.Google Scholar
- QuantLib: A Free/Open-source Library for Quantitative Finance. http://quantlib.org/index.shtml.Google Scholar
- C. P. Sapuntzakis, R. Chandra, B. Pfaff, J. Chow, M. S. Lam, and M. Rosenblum. Optimizing the Migration of Virtual Computers. In Proc. 5th Symposium on Operating Systems Design and Implementation (OSDI), Boston, MA, December 2002. Google ScholarDigital Library
- University of Toronto. SHRiMP: SHort Read Mapping Package. http://compbio.cs.toronto.edu/shrimp/.Google Scholar
- University of Toronto. SnowFlock: Swift VM Cloning for Cloud Computing. http://sysweb.cs.toronto.edu/snowflock.Google Scholar
- M. Steinder, I. Whalley, D. Carrera, I. Gaweda, and D. Chess. Server Virtualization in Autonomic Management of Heterogeneous Workloads. In Proc. 10th Integrated Network Management (IM) conference, Munich, Germany, 2007.Google ScholarCross Ref
- M. Theimer, K. Lantz, and D. Cheriton. Preemptable Remote Execution Facilities for the V-System. In Proc. 10th Symposium on Operating Systems Principles (SOSP), Orcas Island, WA, December 1985. Google ScholarDigital Library
- VMware. VMotion: Migrate Virtual Machines with Zero Downtime. http://www.vmware.com/products/vi/vc/vmotion.html.Google Scholar
- M. Vrable, J. Ma, J. Chen, D. Moore, E. Vandekieft, A. Snoeren, G. Voelker, and S. Savage. Scalability, Fidelity and Containment in the Potemkin Virtual Honeyfarm. In Proc. 20th Symposium on Operating Systems Principles (SOSP), Brighton, UK, October 2005. Google ScholarDigital Library
- C. A. Waldspurger. Memory Resource Management in VMWare ESX Server. In Proc. 5th Symposium on Operating System Design and Implementation (OSDI), Boston, MA, 2002. Google ScholarDigital Library
- A. Warfield, S. Hand, K. Fraser, and T. Deegan. Facilitating the development of soft devices. In Proc. USENIX Annual Technical Conference, Anaheim, CA, April 2005. Google ScholarDigital Library
- T. Wood, P. Shenoy, A. Venkataramani, and M. Yousif. Black-box and Gray-box Strategies for Virtual Machine Migration. In Proc. 4th Symposium on Networked Systems Design and Implementation (NSDI), Cambridge, MA, April 2007. Google ScholarDigital Library
- E. Zayas. Attacking the Process Migration Bottleneck. In Proc. 11th Symposium on Operating System Principles (SOSP), Austin, TX, November 1987. Google ScholarDigital Library
Index Terms
- SnowFlock: rapid virtual machine cloning for cloud computing
Recommendations
SnowFlock: Virtual Machine Cloning as a First-Class Cloud Primitive
A basic building block of cloud computing is virtualization. Virtual machines (VMs) encapsulate a user’s computing environment and efficiently isolate it from that of other users. VMs, however, are large entities, and no clear APIs exist yet to provide ...
Performance Analysis for Pareto-Optimal Green Consolidation Based on Virtual Machines Live Migration
Huge energy requirement of cloud data centers is prime concern. Dynamic Virtual Machine VM consolidation based on VM live migration to switched-off or put some of the under-loaded host Physical Machines PMs into a low power consumption mode can ...
Transparently bridging semantic gap in CPU management for virtualized environments
Consolidated environments are progressively accommodating diverse and unpredictable workloads in conjunction with virtual desktop infrastructure and cloud computing. Unpredictable workloads, however, aggravate the semantic gap between the virtual ...
Comments