Improving Agility and Elasticity in Bare-metal Clouds

Abstract

Bare-metal clouds are an emerging infrastructure-as-a-service (IaaS) that leases physical machines (bare-metal instances) rather than virtual machines, allowing resource-intensive applications to have exclusive access to physical hardware. Unfortunately, bare-metal instances require time-consuming or OS-specific tasks for deployment due to the lack of virtualization layers, thereby sacrificing several beneficial features of traditional IaaS clouds such as agility, elasticity, and OS transparency. We present BMcast, an OS deployment system with a special-purpose de-virtualizable virtual machine monitor (VMM) that supports quick and OS-transparent startup of bare-metal instances. BMcast performs streaming OS deployment while allowing direct access to physical hardware from the guest OS, and then disappears after completing the deployment. Quick startup of instances improves agility and elasticity significantly, and OS transparency greatly simplifies management tasks for cloud customers. Experimental results have confirmed that BMcast initiated a bare-metal instance 8.6 times faster than image copying, and database performance on BMcast during streaming OS deployment was comparable to that on a state-of-the-art VMM without performing deployment. BMcast incurred zero overhead after de-virtualization.

References

1. ELVIS/ELI I/O acceleration code. http://lists.gnu.org/archive/html/qemu-devel/2013-09/msg04610.html.
2. Facebook: Virtualisation does not scale. http://www.zdnet.com/facebook-virtualisation-does-not-scale-4010021998/.
3. Containers at scale. https://speakerdeck.com/jbeda/containers-at-scale.
4. Internap Network Services Corporation. http://www.internap.com/.
5. Liquid Web Inc. http://www.liquidweb.com/.
6. OSU Micro-Benchmarks. http://mvapich.cse.ohio-state.edu/benchmarks/.
7. Openstack. http://www.openstack.org/.
8. Peer 1 Hosting. http://www.peer1.com/.
9. Rackspace, Inc. http://www.rackspace.com/.
10. SoftLayer Technologies, Inc. http://www.softlayer.com/.
11. Sysbench. https://launchpad.net/sysbench.
12. Webair Internet Development, Inc. http://www.webair.com/.
13. fio -- Flexible IO Tester. http://git.kernel.dk/?p=fio.git.
14. ioping. https://code.google.com/p/ioping.
15. ATA over Ethernet Tools. http://aoetools.sourceforge.net/.
16. M. Aron and P. Druschel. Soft Timers: Efficient Microsecond Software Timer Support for Network Processing. ACM Transactions on Computer Systems (TOCS), 18 (3): 197--228, Aug. 2000.
17. P. Barham, B. Dragovic, K. Fraser, S. Hand, T. Harris, A. Ho, R. Neugebauer, I. Pratt, and A. Warfield. Xen and the Art of Virtualization. In Proceedings of the 19th ACM Symposium on Operating Systems Principles, SOSP '03, pages 164--177, Oct. 2003.
18. S. K. Barker and P. Shenoy. Empirical Evaluation of Latency-sensitive Application Performance in the Cloud. In Proceedings of the 1st annual ACM SIGMM Conference on Multimedia Systems, MMSys '10, pages 35--46, Feb. 2010.
19. ach, and Singhania}BarrelfishA. Baumann, P. Barham, P.-E. Dagand, T. Harris, R. Isaacs, S. Peter, T. Roscoe, A. Schüpbach, and A. Singhania. The Multikernel: A New OS Architecture for Scalable Multicore Systems. In Proceedings of the ACM SIGOPS 22nd Symposium on Operating Systems Principles, SOSP '09, pages 29--44, Oct. 2009.
20. M. Ben-Yehuda, M. D. Day, Z. Dubitzky, M. Factor, N. Har'El, A. Gordon, A. Liguori, O. Wasserman, and B.-A. Yassour. The Turtles Project: Design and Implementation of Nested Virtualization. In Proceedings of the 9th USENIX Conference on Operating Systems Design and Implementation, OSDI'10, pages 423--436, Oct. 2010.
21. R. Bhargava, B. Serebrin, F. Spadini, and S. Manne. Accelerating Two-Dimensional Page Walks for Virtualized Systems. In Proceedings of the 13th International Conference on Architectural Support for Programming Languages and Operating Systems, ASPLOS XIII, pages 26--35, Mar. 2008.
22. R. Chandra, N. Zeldovich, C. Sapuntzakis, and M. S. Lam. The Collective: A Cache-Based System Management Architecture. In Proceedings of the 2nd Symposium on Networked Systems Design and Implementation (NSDI), pages 259--272, May 2005.
23. B. F. Cooper, A. Silberstein, E. Tam, R. Ramakrishnan, and R. Sears. Benchmarking Cloud Serving Systems with YCSB. In Proceedings of the 1st ACM symposium on Cloud Computing, SoCC '10, pages 143--154, June 2010.
24. C. David, G.-E. Luis, and R. Sean. OS Streaming Deployment. In Proceedings of the 29th IEEE International Performance Computing and Communications Conference (IPCCC), pages 169 --179, Dec. 2010.
25. K. Elphinstone and G. Heiser. From L3 to seL4 What Have We Learnt in 20 Years of L4 Microkernels? In Proceedings of the 24th ACM Symposium on Operating Systems Principles, SOSP '13, pages 133--150, Nov. 2013.
26. D. R. Engler, M. F. Kaashoek, and J. O'Toole, Jr. Exokernel: An Operating System Architecture for Application-level Resource Management. In Proceedings of the 15th ACM Symposium on Operating Systems Principles, SOSP '95, pages 251--266, Dec. 1995.
27. A. Gordon, N. Amit, N. Har'El, M. Ben-Yehuda, A. Landau, A. Schuster, and D. Tsafrir. ELI: Bare-metal Performance for I/O Virtualization. In Proceedings of the 17th International Conference on Architectural Support for Programming Languages and Operating Systems, ASPLOS XVII, pages 411--422, Mar. 2012.
28. Internap, Inc. Cloud Landscape Report: Price & Performance Obstacles Surface for Fast, Big Data Applications, Jan 2014.
29. A. Iosup, S. Ostermann, N. Yigitbasi, R. Prodan, T. Fahringer, and D. Epema. Performance Analysis of Cloud Computing Services for Many-Tasks Scientific Computing. IEEE Transactions on Parallel and Distributed Systems, 22 (6): 931--945, June 2011.
30. E. Keller, J. Szefer, J. Rexford, and R. B. Lee. NoHype: Virtualized Cloud Infrastructure without the Virtualization. In Proceedings of the 37th Annual International Symposium on Computer Architecture, ISCA '10, pages 350--361, June 2010.
31. A. Kivity. kvm: the Linux Virtual Machine Monitor. In Proceedings of the 2007 Ottawa Linux Symposium, pages 225--230, July 2007.
32. T. Kooburat and M. Swift. The Best of Both Worlds with On-Demand Virtualization. In Proceedings of the 13th Workshop on Hot Topics in Operating Systems, HotOS XIII, May 2011.
33. A. Lakshman and P. Malik. Cassandra: A Decentralized Structured Storage System. ACM SIGOPS Operating Systems Review, 44 (2): 35--40, Apr. 2010.
34. D. E. Lowell, Y. Saito, and E. J. Samberg. Devirtualizable Virtual Machines Enabling General, Single-node, Online Maintenance. In Proceedings of the 11th International Conference on Architectural Support for Programming Languages and Operating Systems, ASPLOS XI, pages 211--223, Oct. 2004.
35. M. Mao and M. Humphrey. A Performance Study on the VM Startup Time in the Cloud. In Proceedings of the IEEE 5th International Conference on Cloud Computing, pages 423--430, June 2012.
36. J. C. Mogul, J. Mudigonda, J. R. Santos, and Y. Turner. The NIC Is the Hypervisor: Bare-Metal Guests in IaaS Clouds. In Proceedings of the 14th USENIX Conference on Hot Topics in Operating Systems, HotOS XIV, May 2013.
37. V. Nae, R. Prodan, T. Fahringer, and A. Iosup. The Impact of Virtualization on the Performance of Massively Multiplayer Online Games. In Proceedings of the 8th Annual Workshop on Network and Systems Support for Games, NetGames '09, page 9, Nov. 2009.
38. S. Ostermann, A. Iosup, N. Yigitbasi, R. Prodan, T. Fahringer, and D. Epema. A Performance Analysis of EC2 Cloud Computing Services for Scientific Computing. In phCloud Computing, volume 34 of Lecture Notes of the Institute for Computer Sciences, Social-Informatics and Telecommunications Engineering, pages 115--131. Springer Berlin Heidelberg, 2010.
39. H. Raj and K. Schwan. High Performance and Scalable I/O Virtualization via Self-Virtualized Devices. In Proceedings of the 16th International Symposium on High Performance Distributed Computing, HPDC '07, pages 179--188, June 2007.
40. K. Razavi and T. Kielmann. Scalable Virtual Machine Deployment Using VM Image Caches. In Proceedings of the International Conference on High Performance Computing, Networking, Storage and Analysis, SC '13, page 65, Nov. 2013.
41. T. Ristenpart, E. Tromer, H. Shacham, and S. Savage. Hey, You, Get off of My Cloud: Exploring Information Leakage in Third-party Compute Clouds. In Proceedings of the 16th ACM Conference on Computer and Communications Security, CCS '09, pages 199--212, 2009.
42. L. Ryzhyk, P. Chubb, I. Kuz, E. Le Sueur, and G. Heiser. Automatic Device Driver Synthesis with Termite. In Proceedings of the ACM SIGOPS 22nd Symposium on Operating Systems Principles, SOSP '09, pages 73--86, Oct. 2009.
43. B. C. S. Hopkins. AoE (ATA over Ethernet). February 2009.
44. A. Seshadri, M. Luk, N. Qu, and A. Perrig. SecVisor: A Tiny Hypervisor to Provide Lifetime Kernel Code Integrity for Commodity OSes. In Proceedings of the 21st ACM Symposium on Operating Systems Principles, SOSP '07, pages 335--350, Oct. 2007.
45. J. Shafer. I/O Virtualization Bottlenecks in Cloud Computing Today. In Proceedings of the Second Workshop on I/O Virtualization, WIOV'10, Mar. 2010.
46. T. Shinagawa, H. Eiraku, K. Tanimoto, K. Omote, S. Hasegawa, T. Horie, M. Hirano, K. Kourai, Y. Oyama, E. Kawai, K. Kono, S. Chiba, Y. Shinjo, and K. Kato. BitVisor: A Thin Hypervisor for Enforcing I/O Device Security. In Proceedings of the 2009 ACM SIGPLAN/SIGOPS International Conference on Virtual Execution Environments, VEE '09, pages 121--130, Mar. 2009.
47. V. Uhlig, J. LeVasseur, E. Skoglund, and U. Dannowski. Towards Scalable Multiprocessor Virtual Machines. In Proceedings of the 3rd Virtual Machine Research And Technology Symposium, pages 43 -- 56, May 2004.
48. G. Wang and T. S. E. Ng. The Impact of Virtualization on Network Performance of Amazon EC2 Data Center. In Proceedings of the 29th IEEE International Conference on Computer Communications, INFOCOM 2010, pages 1163--1171, Mar. 2010.
49. P. Willmann, J. Shafer, D. Carr, A. Menon, S. Rixner, A. L. Cox, and W. Zwaenepoel. Concurrent Direct Network Access for Virtual Machine Monitors. In Proceedings of the IEEE 13th International Symposium on High Performance Computer Architecture, HPCA '07, pages 306--317, Feb. 2007.
50. F. Zhang, J. Chen, H. Chen, and B. Zang. CloudVisor: Retrofitting Protection of Virtual Machines in Multi-tenant Cloud with Nested Virtualization. In Proceedings of the 23rd ACM Symposium on Operating Systems Principles, SOSP '11, pages 203--216, Oct. 2011.

Index Terms

• Computer systems organization

  • Architectures

• Software and its engineering

  • Software creation and management

    • Designing software

  • Software organization and properties

    • Contextual software domains

      • Operating systems