Ryo Nakamura
Hajime Tazaki
ABSTRACT
This paper proposes a novel approach for fast container networking that enables applications in containers to utilize host network stacks directly with proper access control. Our approach, called socket-grafting, offers a new socket-layer communication channel by grafting sockets in containers onto sockets in host network stacks. In contrast to recent approaches that make network stacks faster by using high-speed packet I/O techniques, socket-grafting makes container networking faster by bypassing a container's network stack and virtual interfaces. As a concrete implementation of socket-grafting, we designed AF_GRAFT, which is a new address family for the socket layer. AF_GRAFT performs interconnection between the two socket layers in the container and host network stacks. In this paper, we describe socket-grafting, AF_GRAFT design, and its implementation on Linux. Our evaluation results demonstrate that the proposed method doubled throughput and reduced latency by 23% compared with traditional NAT-based container networking, and improved the network performance of containerized HTTP servers and message queues.
- 2016. XDP - IO Visor Project. (2016). https://www.iovisor.org/technology/xdpGoogle Scholar
- 2017. Intel Data Plane Development Kit. (2017). http://dpdk.org/Google Scholar
- 2017. Linux Containers. (2017). https://linuxcontainers.org/Google Scholar
- 2018. Apache OpenWhisk is a serverless event-based programming service and an Apache Incubator project. (March 2018). https://github.com/apache/incubator-openwhiskGoogle Scholar
- 2018. Cilium. (2018). https://cilium.io/Google Scholar
- 2018. Container Network Interface - networking for Linux containers. (2018). https://github.com/containernetworking/cniGoogle Scholar
- 2018. Kubernetes | Production-Grade Container Orchestration. (2018). https://kubernetes.io/Google Scholar
- 2018. networking:iproute2 {Linux Foundation Wiki}. (Feb. 2018). https://wiki.linuxfoundation.org/networking/iproute2Google Scholar
- 2018. rkt, a security-minded, standards-based container engine. (2018). https://coreos.com/rkt/Google Scholar
- 2018. upa/af-graft. (2018). https://github.com/upa/af-graftGoogle Scholar
- Jake Adriaens. 2017. Google Cloud Platform Blog: Andromeda 2.1 reduces GCP's intra-zone latency by 40%. (Nov. 2017). https://cloudplatform.googleblog.com/2017/11/Andromeda-2-1-reduces-GCPs-intra-zone-latency-by-40-percent.htmlGoogle Scholar
- J. Anderson, H. Hu, U. Agarwal, C. Lowery, H. Li, and A. Apon. 2016. Performance considerations of network functions virtualization using containers. In 2016 International Conference on Computing, Networking and Communications (ICNC). 1--7.Google Scholar
- A. Balalaie, A. Heydarnoori, and P. Jamshidi. 2016. Microservices Architecture Enables DevOps: Migration to a Cloud-Native Architecture. IEEE Software 33, 3 (May 2016), 42--52. Google ScholarDigital Library
- J. Claassen, R. Koning, and P. Grosso. 2016. Linux containers networking: Performance and scalability of kernel modules. In NOMS 2016 - 2016 IEEE/MP Network Operations and Management Symposium. 713--717.Google Scholar
- Jonathan Corbet. 2007. Large receive offload {LWN.net}. (Aug. 2007). https://lwn.net/Articles/243949/Google Scholar
- Hideki Eiraku, Yasushi Shinjo, Calton Pu, Younggyun Koh, and Kazuhiko Kato. 2009. Fast Networking with Socket-outsourcing in Hosted Virtual Machine Environments. In Proceedings of the 2009 ACM Symposium on Applied Computing (SAC '09). ACM, New York, NY, USA, 310--317. Google ScholarDigital Library
- W. Felter, A. Ferreira, R. Rajamony, and J. Rubio. 2015. An updated performance comparison of virtual machines and Linux containers. In 2015 IEEE International Symposium on Performance Analysis of Systems and Software (ISPASS). 171--172.Google Scholar
- Sean Hefty. 2012. RSOCKETS. (March 2012). http://downloads.openfabrics.org/Media/Monterey_2012/2012_Workshop_Mon_Rsockets.pdfGoogle Scholar
- Michio Honda, Felipe Huici, Costin Raiciu, Joao Araujo, and Luigi Rizzo. 2014. Rekindling Network Protocol Innovation with User-level Stacks. SIGCOMM Comput. Commun. Rev. 44, 2 (April 2014), 52--58. Google ScholarDigital Library
- Yang Hu, Mingcong Song, and Tao Li. 2017. Towards "Full Containerization" in Containerized Network Function Virtualization. In Proceedings of the Twenty-Second International Conference on Architectural Support for Programming Languages and Operating Systems (ASPLOS '17). ACM, New York, NY, USA, 467--481. Google ScholarDigital Library
- Junichiro Itoh. 2003. Implementing AF-independent application. (May 2003). http://www.kame.net/newsletter/19980604/Google Scholar
- James Lewis and Martin Fowler. 2014. Microservices. (March 2014). https://martinfowler.com/articles/microservices.htmlGoogle Scholar
- Steven McCanne and Van Jacobson. 1993. The BSD Packet Filter: A New Architecture for User-level Packet Capture. In Proceedings of the USENIX Winter 1993 Conference Proceedings on USENIX Winter 1993 Conference Proceedings (USENIX'93). USENIX Association, Berkeley, CA, USA, 2--2. http://dl.acm.org/citation.cfm?id=1267303.1267305 Google ScholarDigital Library
- Dirk Merkel. 2014. Docker: Lightweight Linux Containers for Consistent Development and Deployment. Linux Journal. 2014, 239, Article 2 (March 2014). http://dl.acm.org/citation.cfm?id=2600239.2600241 Google ScholarDigital Library
- R. Morabito, J. KjÃd'llman, and M. Komu. 2015. Hypervisors vs. Lightweight Virtualization: A Performance Comparison. In 2015 IEEE International Conference on Cloud Engineering. 386--393. Google ScholarDigital Library
- Cristina Opriceana and Hajime Tazaki.2017. Network stack personality in Android phone. (2017). https://www.netdevconf.org/2.2/papers/tazaki-mptcp-talk.pdfGoogle Scholar
- Simon Peter, Jialin Li, Irene Zhang, Dan R. K. Ports, Doug Woos, Arvind Krishnamurthy, Thomas Anderson, and Timothy Roscoe. 2014. Arrakis: The Operating System is the Control Plane. In 11th USENIX Symposium on Operating Systems Design and Implementation (OSDI 14). USENIX Association, CO, 1--16. https://www.usenix.org/conference/osdi14/technical-sessions/presentation/peter Google ScholarDigital Library
- Kevin Pulo. 2009. Fun with LD_PRELOAD. (Jan. 2009). https://linux.org.au/conf/2009/slides/172.pdfGoogle Scholar
- Luigi Rizzo. 2012. netmap: A Novel Framework for Fast Packet I/O. In 2012 USENIX Annual Technical Conference (USENIX ATC 12). USENIX Association, Boston, MA, 101--112. https://www.usenix.org/conference/atc12/technical-sessions/presentation/rizzo Google ScholarDigital Library
- Jay Schulist, Daniel Borkmann, and Alexei Starovoitov. 2018. Linux Socket Filtering aka Berkeley Packet Filter (BPF). (Jan. 2018). https://www.kernel.org/doc/Documentation/networking/filter.txtGoogle Scholar
- Amazon Web Services. 2018. AWS Lambda - Serverless Compute. (2018). https://aws.amazon.com/lambda/?nc1=h_lsGoogle Scholar
- Solarflare. 2013. OpenOnload. (2013). http://www.openonload.org/Google Scholar
- Dinesh Subhraveti, Sri Goli, Serge Hallyn, Ravi Chamarthy, and Christos Kozyrakis. 2017. AppSwitch: Resolving the Application Identity Crisis. (2017). arXiv:arXiv:1711.02294Google Scholar
- Mellanox Technologies. 2018. Mellanox/sockperf: Network Benchmarking Utility. (March 2018). https://github.com/Mellanox/sockperfGoogle Scholar
- Kenichi Yasukata, Michio Honda, Douglas Santry, and Lars Eggert. 2016. StackMap: Low-Latency Networking with the OS Stack and Dedicated NICs. In 2016 USENIX Annual Technical Conference (USENIX ATC 16). USENIX Association, Denver, CO. https://www.usenix.org/conference/atc16/technical-sessions/presentation/stackmap-low-latency-networking-os-stack-and Google ScholarDigital Library
- Tianlong Yu, Shadi Abdollahian Noghabi, Shachar Raindel, Hongqiang Liu, Jitu Padhye, and Vyas Sekar. 2016. FreeFlow: High Performance Container Networking. In Proceedings of the 15th ACM Workshop on Hot Topics in Networks (HotNets '16). ACM, New York, NY, USA, 43--49. Google ScholarDigital Library
- Wei Zhang, Guyue Liu, Wenhui Zhang, Neel Shah, Phillip Lopreiato, Gregoire Todeschi, K.K. Ramakrishnan, and Timothy Wood. 2016. OpenNetVM: A Platform for High Performance Network Service Chains. In Proceedings of the 2016 Workshop on Hot Topics in Middleboxes and Network Function Virtualization (HotMIddlebox '16). ACM, New York, NY, USA, 26--31. https://doi.org/2940147.2940155 Google ScholarCross Ref
- Yang Zhao, Nai Xia, Chen Tian, Bo Li, Yizhou Tang, Yi Wang, Gong Zhang, Rui Li, and Alex X. Liu. 2017. Performance of Container Networking Technologies. In Proceedings of the Workshop on Hot Topics in Container Networking and Networked Systems (HotConNet '17). ACM, New York, NY, USA, 1--6. Google ScholarDigital Library
Index Terms
- Grafting sockets for fast container networking
Recommendations
Automated Multi-Swarm Networking with Open Baton NFV MANO Framework
Euro-Par 2018: Parallel Processing WorkshopsAbstractContainer-based Network Functions Virtualization (NFV) and multi-site/multi-cluster service orchestration are a critical topic in the field of ICT infrastructure. Academia, Industry and Open Source projects are actively working on the technology. ...
Analysis on container port capacity: a Markovian modeling approach
Container ports handle outbound, inbound, and transshipment containers plying between the area for vessels on the quay and the storage space in the yard. Port operators typically concentrate their efforts on the container handling process with the aims ...
Container stowage pre-planning: using search to generate solutions, a case study
Container-ships are vessels possessing an internal structure that facilitates the handling of containerised cargo. At each port along the vessel's journey, containers destined for those ports are unloaded and additional containers destined for ...
Comments