Moving beyond end-to-end path information to optimize CDN performance

ABSTRACT

Replicating content across a geographically distributed set of servers and redirecting clients to the closest server in terms of latency has emerged as a common paradigm for improving client performance. In this paper, we analyze latencies measured from servers in Google's content distribution network (CDN) to clients all across the Internet to study the effectiveness of latency-based server selection. Our main result is that redirecting every client to the server with least latency does not suffice to optimize client latencies. First, even though most clients are served by a geographically nearby CDN node, a sizeable fraction of experience latencies several tens of milliseconds higher than other in the same region. Second, we find that queueing delays often override the benefits of a client interacting with a nearby server.

To help the administrators of Google's CDN cope with these problems, we have built a system called WhyHigh. First, WhyHigh measures client latencies across all nodes in the CDN and correlates measurements to identify the prefixes affected by inflated latencies. Second, since clients in several thousand prefixes have poor latencies, WhyHigh prioritizes problems based on the impact that solving them would have, e.g., by identifying either an AS path common to several inflated prefixes or a CDN node where path inflation is widespread. Finally, WhyHigh diagnoses the causes for inflated latencies using active measurements such as traceroutes and pings, in combination with datasets such as BGP paths and flow records. Typical causes discovered include lack of peering, routing misconfigurations, and side-effects of traffic engineering. We have used WhyHigh to diagnose several instances of inflated latencies, and our efforts over the course of a year have significantly helped improve the performance offered to clients by Google's CDN.

References

1. Akamai, Inc. home page. http://www.akamai.com.
2. D. G. Andersen, H. Balakrishnan, M. F. Kaashoek, and R. Morris. Resilient overlay networks. In SOSP, 2001.
3. N. Cardwell, S. Savage, and T. Anderson. Modeling TCP latency. In INFOCOM, 2000.
4. M. Dischinger, A. Haeberlen, K. P. Gummadi, and S. Saroiu. Characterizing residential broadband networks. In IMC, 2007.
5. A. Feldmann, O. Maennel, Z. M. Mao, A. Berger, and B. Maggs. Locating Internet routing instabilities. In SIGCOMM, 2004.
6. M. J. Freedman, E. Freudenthal, and D. Mazières. Democratizing content publication with Coral. In NSDI, 2004.
7. M. J. Freedman, K. Lakshminarayanan, and D. Mazieres. OASIS: Anycast for any service. In NSDI, 2006.
8. Y. He, M. Faloutsos, S. Krishnamurthy, and B. Huffaker. On routing asymmetry in the Internet. In Autonomic Networks Symposium in Globecom, 2005.
9. C. Huang, A. Wang, J. Li, and K. W. Ross. Measuring and evaluating large-scale CDNs. In IMC, 2008.
10. E. Katz-Bassett, J. P. John, A. Krishnamurthy, D. Wetherall, T. Anderson, and Y. Chawathe. Towards IP geolocation using delay and topology measurements. In IMC, 2006.
11. E. Katz-Bassett, H. V. Madhyastha, J. P. John, A. Krishnamurthy, and T. Anderson. Studying black holes in the Internet with Hubble. In NSDI, 2008.
12. H. V. Madhyastha, T. Isdal, M. Piatek, C. Dixon, T. Anderson, A. Krishnamurthy, and A. Venkataramani. iPlane: An information plane for distributed services. In OSDI, 2006.
13. H. V. Madhyastha, E. Katz-Bassett, T. Anderson, A. Krishnamurthy, and A. Venkataramani. iPlane Nano: Path prediction for peer-to-peer applications. In NSDI, 2009.
14. R. Mahajan. Practical and Efficient Internet Routing with Competing Interests. PhD thesis, University of Washington, 2005.
15. R. Mahajan, M. Zhang, L. Poole, and V. Pai. Uncovering performance differences among backbone ISPs with Netdiff. In NSDI, 2008.
16. V. Paxson. End-to-end routing behavior in the Internet. ToN, 1997.
17. Reverse traceroute. http://www.cs.washington.edu/research/networking/astronomy/reverse-traceroute.html.
18. S. Savage, T. Anderson, A. Aggarwal, D. Becker, N. Cardwell, A. Collins, E. Hoffman, J. Snell, A. Vahdat, G. Voelker, and J. Zahorjan. Detour: a case for informed Internet routing and transport. IEEE Micro, 1999.
19. N. Spring, R. Mahajan, and T. Anderson. Quantifying the causes of path inflation. In SIGCOMM, 2003.
20. N. Spring, R. Mahajan, D. Wetherall, and T. Anderson. Measuring ISP topologies with Rocketfuel. ToN, 2004.
21. M. Tariq, A. Zeitoun, V. Valancius, N. Feamster, and M. Ammar. Answering what-if deployment and configuration questions with WISE. In SIGCOMM, 2008.
22. J. Wu, Z. M. Mao, and J. R. J. Wang. Finding a needle in a haystack: Pinpointing significant BGP routing changes in an IP network. In NSDI, 2005.
23. M. Zhang, C. Zhang, V. Pai, L. Peterson, and R. Wang. PlanetSeer: Internet path failure monitoring and characterization in wide-area services. In OSDI, 2004.
24. Y. Zhang, Z. M. Mao, and M. Zhang. Effective diagnosis of routing disruptions from end systems. In NSDI, 2008.
25. Y. Zhang, V. Paxson, and S. Shenker. The stationarity of Internet path properties: Routing, loss, and throughput. Technical report, ACIRI, 2000.

Index Terms

• Networks

  • Network properties

    • Network range

      • Local area networks

      • Wide area networks

  • Network services

    • Network management

    • Network monitoring