This thesis describes the design and implementation of a high performance network interface chip called the APIC (ATM Port Interconnect Controller). It also describes architectural enhancements to operating system (OS) software that are necessary to exploit some of the novel features that have been integrated into this chip.
One of the ways in which the APIC addresses the memory bottleneck is to function in a desk-area environment where different memories can be used to spread the load. The idea here is to dedicate one APIC chip and one memory bank to each high-bandwidth device in the system, thereby shedding the load from a host system's main memory. Several such APIC-memory-device combinations can be daisy chained to form a desk-area network with high bandwidth and low latency characteristics.
There are several well-known operating system overheads associated with in-kernel implementations of network interface device drivers. These include context switch latency, system call overhead, and interrupt over-head. It is possible to remove a number of these inefficiencies and allow for increased performance for end applications if the data path of the device driver can be implemented as a library in user-space.
Another problem, is receive livelock; this term is used to describe the situation in which, under heavy load, an operating system servicing a device might end up spending all its time in the interrupt service routine, and no useful work gets done. The APIC introduces a novel concept called Interrupt Demultiplexing, which taken alone can alleviate the effects of interrupt livelock, but in conjunction with user-space drivers can solve the problem entirely.
Network interfaces, except for ATM interfaces, have traditionally not provided special mechanisms for supporting quality-of-service (QoS) guarantees. Even ATM interfaces have traditionally supported QoS only to a limited extent. By providing pacing support independently for large numbers of connections, the APIC is able to efficiently and reliably support QoS guarantees simultaneously for large numbers of multimedia streams. This can be especially useful in the context of large multimedia-on-demand servers. This feature was made possible through a novel pacer design which uses a hardware d-heap data structure. (Abstract shortened by UMI.)
Cited By
- Ousterhout J, Agrawal P, Erickson D, Kozyrakis C, Leverich J, Mazières D, Mitra S, Narayanan A, Ongaro D, Parulkar G, Rosenblum M, Rumble S, Stratmann E and Stutsman R (2011). The case for RAMCloud, Communications of the ACM, 54:7, (121-130), Online publication date: 1-Jul-2011.
- Ousterhout J, Agrawal P, Erickson D, Kozyrakis C, Leverich J, Mazières D, Mitra S, Narayanan A, Parulkar G, Rosenblum M, Rumble S, Stratmann E and Stutsman R (2010). The case for RAMClouds, ACM SIGOPS Operating Systems Review, 43:4, (92-105), Online publication date: 27-Jan-2010.
Recommendations
The APIC Approach to High Performance Network Interface Design: Protected DMA and Other Techniques
INFOCOM '97: Proceedings of the INFOCOM '97. Sixteenth Annual Joint Conference of the IEEE Computer and Communications Societies. Driving the Information RevolutionWe are building a high performance 1.2 Gb/s ATM network interface chip called the APIC (ATM Port Interconnect Controller). In addition to borrowing useful ideas from a number of research and commercial prototypes, the APIC design embraces several ...
Building a high-performance communication layer over virtual interface architecture on Linux clusters
ICS '01: Proceedings of the 15th international conference on SupercomputingThe Virtual Interface Architecture (VIA) is an industry standard user-level communication architecture for cluster or system area networks. The VIA provides a protected, directly-accessible interface to a network hardware, removing the operating system ...