Supporting soft real-time tasks in the xen hypervisor

ABSTRACT

Virtualization technology enables server consolidation and has given an impetus to low-cost green data centers. However, current hypervisors do not provide adequate support for real-time applications, and this has limited the adoption of virtualization in some domains. Soft real-time applications, such as media-based ones, are impeded by components of virtualization including low-performance virtualization I/O, increased scheduling latency, and shared-cache contention. The virtual machine scheduler is central to all these issues. The goal in this paper is to adapt the virtual machine scheduler to be more soft-real-time friendly.

We improve two aspects of the VMM scheduler -- managing scheduling latency as a first-class resource and managing shared caches. We use enterprise IP telephony as an illustrative soft real-time workload and design a scheduler S that incorporates the knowledge of soft real-time applications in all aspects of the scheduler to support responsiveness. For this we first define a laxity value that can be interpreted as the target scheduling latency that the workload desires. The load balancer is also designed to minimize the latency for real-time tasks. For cache management, we take cache-affinity into account for real time tasks and load-balance accordingly to prevent cache thrashing. We measured cache misses and demonstrated that cache management is essential for soft real time tasks. Although our scheduler S employs a different design philosophy, interestingly enough it can be implemented with simple modifications to the Xen hypervisor's credit scheduler. Our experiments demonstrate that the Xen scheduler with our modifications can support soft real-time guests well, without penalizing non-real-time domains.

References

1. P. Barham, B. Dragovic, K. Fraser, S. Hand, T. Harris, A. Ho, R. Neugebauer, I. Pratt and A. Warfield, "Xen and the art of virtualization," in ACM SOSP 2003.
2. D. Patnaik, A.S. Krishnakumar, P. Krishnan, N. Singh, S. Yajnik, "Performance implications of hosting enterprise telephony applications on virtualized multi-core platforms," in IPTComm 2009.
3. N. Nishiguchi, "Evaluation and consideration of the credit scheduler for client virtualization," Xen Summit Asia 2008.
4. D. Ongaro, A. L. Cox, and S. Rixner, "Scheduling I/O in virtual machine monitors," in VEE '08: Proceedings of the Fourth ACM SIGPLAN/SIGOPS International Conference on Virtual Execution Environments, pp. 1--10, 2008.
5. G. Liao, D. Guo, L. Bhuyan, and S. R. King, "Software techniques to improve virtualized IO performance on multi-core systems," in ANCS '08: Proceedings of the 4th ACM/IEEE Symposium on Architectures for Networking and Communications Systems, pp. 161--170, 2008.
6. E. Ackaouy, "New CPU scheduler with SMP load balancer," http://lists.xensource.com/archives/html/xen--devel/2006--05/msg01315.htm
7. A. Menon, J. R. Santos, Y. Turner, G. J. Janakiraman, and W. Zwaenepoel, "Diagnosing performance overheads in the Xen virtual machine environment," in VEE '05: Proceedings of the 1st ACM/USENIX International Conference on Virtual execution environments, pp. 13--23, 2005.
8. SIPp, http://sipp.sourceforge.net/
9. PESQ, http://www.itu.int/rec/T-REC-P.862/en, ITU-T Recommendation P.862, "Perceptual evaluation of speech quality (PESQ): An objective method for end-to-end speech quality assessment of narrow-band telephone networks and speech codecs".
10. P. Apparao, R. Iyer, X. Zhang, D. Newell, and T. Adelmeyer, "Characterization & analysis of a server consolidation benchmark," in VEE '08:Proceedings of the fourth ACM SIGPLAN/SIGOPS international conference on Virtual Execution Environments, pp. 21--30, 2008.
11. A. Menon, A. L. Cox, and W. Zwaenepoel, "Optimizing network virtualization in Xen," in ATEC '06: Proceedings of the USENIX '06 Annual Technical Conference, 2006.
12. S. Govindan, A. R. Nath, A. Das, B. Urgaonkar, and A. Sivasubramaniam, "Xen and co.: communication-aware CPU scheduling for consolidated Xen-based hosting platforms," in VEE '07: Proceedings of the 3rd International Conference on Virtual Execution Environments, pp.126--136, 2007.
13. SEDF Scheduler: Xen wikipage on Xen scheduling, http://wiki.xensource.com/xenwiki/Scheduling.
14. G. Dunlap, Planned csched improvements and credit2, Xen mailing list, http://lists.xensource.com/archives/html/xen-devel/2009-10/msg00442.html
15. K. Duda and D. Cheriton, "Borrowed-virtual-time (BVT) scheduling: supporting latency-sensitive threads in a general purpose scheduler," in ACM SIGOPS Operating Systems Review, 33 (5), December 1999.
16. I. Stoica, H. Abdel-Wahab, K. Jeffay, S. Baruah, J. Gehrke, G. Plaxton, "A proportional share resource allocation algorithm for real-time, time-shared systems," Proc. 17th IEEE Real Time System Symposium, December 1996.
17. J. Nieh and M. Lam, "A SMART scheduler for multimedia applications," in ACM Transactions on Computer Systems, vol. 21, no. 2, May 2003, pp. 117--163.
18. P. Goyal, H. Vin, H. Chen, "Start-time fair queuing: A scheduling algorithm for integrated service packet switching networks," in Proc. of the ACM SIGCOMM Conf. on Applications, Technologies, Architectures, and Protocols for Computer Communications, August 1996.
19. H. Kim, H. Lim, J. Jeong, H. Jo, J. Lee, "Task-aware virtual machine scheduling for I/O performance," VEE 2009, pp. 101--110.
20. X. Zhang and Y. Dong, "Optimizing Xen VMM based on Intel virtualization technology," in ICICSE '08: Proceedings of the 2008 International Conference on Internet Computing in Science and Engineering, pp. 367--374, 2008.
21. P. Apparao, S. Makineni, and D. Newell, "Characterization of network processing overheads in Xen," in VTDC '06: Proceedings of the 2nd International Workshop on Virtualization Technology in Distributed Computing, p.2, 2006.
22. H. Oi and F. Nakajima, "Performance analysis of large receive offload in a Xen virtualized system," International Conference on Computer Engineering and Technology, vol. 1, pp. 375--480, 2009.
23. P. Willmann, J. Shafer, D. Carr, A. Menon, S. Rixner, A. L. Cox, and W. Zwaenepoel, "Concurrent direct network access for virtual monitors," in HPCA '07: Proceedings of the 2007 IEEE 13th International Symposium on High Performance Computer Architecture, pp. 306--317, 2007.
24. J. R. Santos, Y. Turner, G. Janakiraman, and I. Pratt, "Bridging the gap between software and hardware techniques for IO virtualization," in ATC'08: USENIX 2008 Annual Technical Conference on Annual Technical Conference, pp. 29--42, 2008.
25. H. Raj and K. Schwan, "High performance and scalable IO virtualization via self-virtualized devices," in HPDC '07: Proceedings of the 16th International Symposium on High Performance Distributed Computing, pp. 179--188, ACM, 2007.
26. J. Liu, W. Huang, B. Abali, and D. K. Panda, "High performance VMM-bypass I/O in virtual machines," in ATEC '06: Proceedings of the Annual Conference on USENIX '06 Annual Technical Conference, 2006.
27. S. Thibault, "Stub domains," in Xen Summit, June 2008.

Index Terms

• Software and its engineering

  • Software organization and properties

    • Contextual software domains

      • Operating systems

        • Process management

          • Scheduling