Accurately characterizing end-to-end Internet dynamics is exceptionally difficult, due to the network's immense heterogeneity. It can be impossible to gauge the generality of findings based on measurements of a handful of paths, yet logistically it has proven very difficult to obtain end-to-end measurements on larger scales.
At the heart of our work is a "measurement framework" we devised in which a number of sites around the Internet host a specialized measurement service. By coordinating "probes" between pairs of these sites we can measure end-to-end behavior along $O(N\sp2)$ paths for a framework consisting of N sites. Consequently, we obtain a superlinear scaling that allows us to measure a rich cross-section of Internet behavior without requiring huge numbers of observation points. 37 sites participated in our study, allowing us to measure more than 1,000 distinct Internet paths.
The first part of our work looks at the behavior of end-to-end routing: the series of routers over which a connection's packets travel. We find that pathologies increased significantly over the course of 1995, indicating that, by one metric, routing degraded over the year; that Internet paths are heavily dominated by a single route, but that routing lifetimes range from seconds to many days, with most lasting for days; and that, at the end of 1995, about half of all Internet paths included a major routing asymmetry.
The second part of our work studies end-to-end Internet packet dynamics.
To cope with such large-scaled measurements requires attention to calibration using self-consistency checks; robust statistics to avoid skewing by outliers; and automated "micro-analysis," such as that performed by tcpanaly, that we might see the forest as well as the trees.
With due diligence to remove packet filter errors and TCP effects, TCP-based measurement provides a viable means for assessing end-to-end packet dynamics.
We find wide ranges of behavior, so we must exercise great caution in regarding any aspect of packet dynamics as "typical."
Some common assumptions such as in-order packet delivery, FIFO bottleneck queueing, independent loss events, single congestion time scales, and path symmetries are all sometimes violated.
The combination of path asymmetries and reverse-path noise render sender-only measurements techniques markedly inferior to those that include receiver-cooperation. (Abstract shortened by UMI.)
Cited By
- Sendra S, Lloret J, Lacuesta R and Jimenez J (2019). Energy Efficiency in Cooperative Wireless Sensor Networks, Mobile Networks and Applications, 24:2, (678-687), Online publication date: 1-Apr-2019.
- Mani S, Durairajan R, Barford P and Sommers J MNTP Proceedings of the 2016 Internet Measurement Conference, (335-348)
- Ding H and Rabinovich M (2015). TCP Stretch Acknowledgements and Timestamps, ACM SIGCOMM Computer Communication Review, 45:3, (20-27), Online publication date: 13-Jul-2015.
- Durairajan R, Mani S, Sommers J and Barford P Time's Forgotten Proceedings of the 14th ACM Workshop on Hot Topics in Networks, (1-7)
- Lu N, Wang Y, Su S and Yang F (2014). A novel path-based approach for single-packet IP traceback, Security and Communication Networks, 7:2, (309-321), Online publication date: 1-Feb-2014.
- Durairajan R, Sommers J and Barford P Layer 1-informed Internet Topology Measurement Proceedings of the 2014 Conference on Internet Measurement Conference, (381-394)
- Raghavendra R, Kaminsky M, Papagiannaki K, Seshan S and Belding E (2011). IdleChat, ACM SIGMOBILE Mobile Computing and Communications Review, 15:3, (10-22), Online publication date: 29-Nov-2011.
- Chan E, Chen A, Luo X, Mok R, Li W and Chang R TRIO Proceedings of the Seventh COnference on emerging Networking EXperiments and Technologies, (1-12)
- Ramachandran G, Ciavattone L and Morton A Restoration measurements on an IP/MPLS backbone Proceedings of the Nineteenth International Workshop on Quality of Service, (1-6)
- Popescu A and Constantinescu D On kleinrock's independence assumption Network performance engineering, (1-13)
- Palmieri P and Pereira O Implementing information-theoretically secure oblivious transfer from packet reordering Proceedings of the 14th international conference on Information Security and Cryptology, (332-345)
- Oliver E Characterizing the transport behaviour of the short message service Proceedings of the 8th international conference on Mobile systems, applications, and services, (223-238)
- Sommers J, Barford P, Duffield N and Ron A (2010). Multiobjective monitoring for SLA compliance, IEEE/ACM Transactions on Networking (TON), 18:2, (652-665), Online publication date: 1-Apr-2010.
- Liebeherr J, Fidler M and Valaee S (2010). A system-theoretic approach to bandwidth estimation, IEEE/ACM Transactions on Networking (TON), 18:4, (1040-1053), Online publication date: 1-Aug-2010.
- Mirza M, Sommers J, Barford P and Zhu X (2010). A machine learning approach to TCP throughput prediction, IEEE/ACM Transactions on Networking (TON), 18:4, (1026-1039), Online publication date: 1-Aug-2010.
- Crosby S, Wallach D and Riedi R (2009). Opportunities and Limits of Remote Timing Attacks, ACM Transactions on Information and System Security (TISSEC), 12:3, (1-29), Online publication date: 1-Jan-2009.
- Chan E, Luo X and Chang R A minimum-delay-difference method for mitigating cross-traffic impact on capacity measurement Proceedings of the 5th international conference on Emerging networking experiments and technologies, (205-216)
- Crotti M, Gringoli F and Salgarelli L Impact of asymmetric routing on statistical traffic classification Proceedings of the 28th IEEE conference on Global telecommunications, (655-662)
- Harfoush K, Bestavros A and Byers J (2009). Measuring capacity bandwidth of targeted path segments, IEEE/ACM Transactions on Networking (TON), 17:1, (80-92), Online publication date: 1-Feb-2009.
- Lee S, Ko Y, Lee D and Hyun S A nondisruptive adaptation scheme for peer-to-peer live video streaming during vertical handoff Proceedings of the 6th IEEE Conference on Consumer Communications and Networking Conference, (138-139)
- Bash B Informed detour selection helps reliability Proceedings of the 28th IEEE international conference on Computer Communications Workshops, (43-48)
- Ngamwongwattana B and Sombun A Enhancing adaptive control in VOIP by caracterizing congestive packet loss Proceedings of the 11th international conference on Advanced Communication Technology - Volume 3, (1847-1851)
- Li M and Zhao W Measurement-based findings in teletraffic (I) Proceedings of the 8th WSEAS international conference on Instrumentation, measurement, circuits and systems, (19-24)
- Mota-Garcia E and Hasimoto-Beltran R Clock offset estimation using collaborative one-way transit time Proceedings of the 2008 ACM SIGMETRICS international conference on Measurement and modeling of computer systems, (455-456)
- Mota-Garcia E and Hasimoto-Beltran R (2008). Clock offset estimation using collaborative one-way transit time, ACM SIGMETRICS Performance Evaluation Review, 36:1, (455-456), Online publication date: 12-Jun-2008.
- Henke C, Schmoll C and Zseby T (2008). Empirical evaluation of hash functions for multipoint measurements, ACM SIGCOMM Computer Communication Review, 38:3, (39-50), Online publication date: 1-Jul-2008.
- Qiu T, Ni J, Wang H, Hua N, Yang Y and Xu J Packet doppler Proceedings of the 2008 ACM CoNEXT Conference, (1-12)
- Perälä P, Jurvansuu M and Prokkola J Combined terminal and network measurement system for bottleneck localization Proceedings of the 4th International Conference on Testbeds and research infrastructures for the development of networks & communities, (1-7)
- Pucha H, Zhang Y, Mao Z and Hu Y Understanding network delay changes caused by routing events Proceedings of the 2007 ACM SIGMETRICS international conference on Measurement and modeling of computer systems, (73-84)
- Mirza M, Sommers J, Barford P and Zhu X A machine learning approach to TCP throughput prediction Proceedings of the 2007 ACM SIGMETRICS international conference on Measurement and modeling of computer systems, (97-108)
- Pucha H, Zhang Y, Mao Z and Hu Y (2007). Understanding network delay changes caused by routing events, ACM SIGMETRICS Performance Evaluation Review, 35:1, (73-84), Online publication date: 12-Jun-2007.
- Mirza M, Sommers J, Barford P and Zhu X (2019). A machine learning approach to TCP throughput prediction, ACM SIGMETRICS Performance Evaluation Review, 35:1, (97-108), Online publication date: 12-Jun-2007.
- Sommers J, Barford P, Duffield N and Ron A Accurate and efficient SLA compliance monitoring Proceedings of the 2007 conference on Applications, technologies, architectures, and protocols for computer communications, (109-120)
- Sommers J, Barford P, Duffield N and Ron A (2007). Accurate and efficient SLA compliance monitoring, ACM SIGCOMM Computer Communication Review, 37:4, (109-120), Online publication date: 1-Oct-2007.
- Kim D and Lee J End-to-end one-way delay estimation using one-way delay variation and round-trip time The Fourth International Conference on Heterogeneous Networking for Quality, Reliability, Security and Robustness & Workshops, (1-8)
- Liebeherr J, Fidler M and Valaee S A Min-Plus System Interpretation of Bandwidth Estimation Proceedings of the IEEE INFOCOM 2007 - 26th IEEE International Conference on Computer Communications, (1127-1135)
- Sommers J, Barford P, Duffield N and Ron A A Framework for Multi-Objective SLA Compliance Monitoring Proceedings of the IEEE INFOCOM 2007 - 26th IEEE International Conference on Computer Communications, (2446-2450)
- Begin T, Brandwajn A, Baynat B, Wolfinger B and Fdida S Towards an automatic modeling tool for observed system behavior Proceedings of the 4th European performance engineering conference on Formal methods and stochastic models for performance evaluation, (200-212)
- Kim J, Park J, Kim S, Lee D, Kim Y and Jin S (2006). Customer-centric service quality management for SK telecom's mobile packet service, International Journal of Network Management, 16:6, (413-427), Online publication date: 1-Nov-2006.
- Rewaskar S, Kaur J and Smith F (2019). A passive state-machine approach for accurate analysis of TCP out-of-sequence segments, ACM SIGCOMM Computer Communication Review, 36:3, (51-64), Online publication date: 5-Jul-2006.
- Paxson V (2006). End-to-end routing behavior in the internet, ACM SIGCOMM Computer Communication Review, 36:5, (41-56), Online publication date: 10-Oct-2006.
- Moser S and Martin J Assessing denial of service vulnerabilities in DOCSIS Proceedings of the 44th annual Southeast regional conference, (337-342)
- Roychoudhuri L, Al-Shaer E and Brewster G (2006). On the impact of loss and delay variation on Internet packet audio transmission, Computer Communications, 29:10, (1578-1589), Online publication date: 1-Jun-2006.
- Kurant M, Nguyen H and Thiran P Survey on dependable IP over fiber networks Dependable Systems, (55-81)
- Peláez-Moreno C, Gallardo-Antolín A, Gómez-Cajas D and Díaz-de-María F (2006). A comparison of front-ends for bitstream-based ASR over IP, Signal Processing, 86:7, (1502-1508), Online publication date: 1-Jul-2006.
- Bi J, Wu Q and Li Z (2006). On estimating clock skew for one-way measurements, Computer Communications, 29:8, (1213-1225), Online publication date: 1-May-2006.
- Lahmadi A, Andrey L and Festor O On delays in management frameworks Proceedings of the 17th IFIP/IEEE international conference on Distributed Systems: operations and management, (13-24)
- Noh M, Gil J, Yoo K and Ahn S A study on L2/OPN design for grid high performance network Proceedings of the 2006 international conference on Computational Science and Its Applications - Volume Part II, (1169-1178)
- Yu K, Lee W, Ahn S and Chung J Automatic location detection system for anomaly traffic on wired/wireless networks Proceedings of the 2006 international conference on Computational Science and Its Applications - Volume Part II, (982-988)
- Bestavros A, Byers J and Harfoush K (2005). Inference and Labeling of Metric-Induced Network Topologies, IEEE Transactions on Parallel and Distributed Systems, 16:11, (1053-1065), Online publication date: 1-Nov-2005.
- Roughan M Fundamental bounds on the accuracy of network performance measurements Proceedings of the 2005 ACM SIGMETRICS international conference on Measurement and modeling of computer systems, (253-264)
- Mao Z, Qiu L, Wang J and Zhang Y On AS-level path inference Proceedings of the 2005 ACM SIGMETRICS international conference on Measurement and modeling of computer systems, (339-349)
- Roughan M (2005). Fundamental bounds on the accuracy of network performance measurements, ACM SIGMETRICS Performance Evaluation Review, 33:1, (253-264), Online publication date: 6-Jun-2005.
- Mao Z, Qiu L, Wang J and Zhang Y (2005). On AS-level path inference, ACM SIGMETRICS Performance Evaluation Review, 33:1, (339-349), Online publication date: 6-Jun-2005.
- Choi B, Moon S, Cruz R, Zhang Z and Diot C Practical delay monitoring for ISPs Proceedings of the 2005 ACM conference on Emerging network experiment and technology, (83-92)
- Tariq M, Dhamdhere A, Dovrolis C and Ammar M Poisson versus periodic path probing (or, does PASTA matter?) Proceedings of the 5th ACM SIGCOMM conference on Internet measurement, (10-10)
- Andersen D, Balakrishnan H, Kaashoek M and Rao R Improving web availability for clients with MONET Proceedings of the 2nd conference on Symposium on Networked Systems Design & Implementation - Volume 2, (115-128)
- Freedman M, Lakshminarayanan K, Rhea S and Stoica I Non-transitive connectivity and DHTs Proceedings of the 2nd conference on Real, Large Distributed Systems - Volume 2, (55-60)
- Yin J, Lv S, Cai Z and Liu C On a novel filtering mechanism for capacity estimation Proceedings of the First Asian Internet Engineering conference on Technologies for Advanced Heterogeneous Networks, (269-281)
- Li M Derivations of error bound on recording traffic time series with long-range dependence Proceedings of the 2005 international conference on Advances in Intelligent Computing - Volume Part I, (360-369)
- Piratla N, Jayasumana A and Bare A Reorder density (RD) Proceedings of the 4th IFIP-TC6 international conference on Networking Technologies, Services, and Protocols; Performance of Computer and Communication Networks; Mobile and Wireless Communication Systems, (78-89)
- Hohn N, Veitch D, Papagiannaki K and Diot C Bridging router performance and queuing theory Proceedings of the joint international conference on Measurement and modeling of computer systems, (355-366)
- Hohn N, Veitch D, Papagiannaki K and Diot C (2019). Bridging router performance and queuing theory, ACM SIGMETRICS Performance Evaluation Review, 32:1, (355-366), Online publication date: 1-Jun-2004.
- Kapoor R, Chen L, Lao L, Gerla M and Sanadidi M CapProbe Proceedings of the 2004 conference on Applications, technologies, architectures, and protocols for computer communications, (67-78)
- Sommers J and Barford P Self-configuring network traffic generation Proceedings of the 4th ACM SIGCOMM conference on Internet measurement, (68-81)
- Katti S, Katabi D, Blake C, Kohler E and Strauss J MultiQ Proceedings of the 4th ACM SIGCOMM conference on Internet measurement, (245-250)
- Brik V, Stroik J and Banerjee S Debugging DHCP performance Proceedings of the 4th ACM SIGCOMM conference on Internet measurement, (257-262)
- Paxson V Strategies for sound internet measurement Proceedings of the 4th ACM SIGCOMM conference on Internet measurement, (263-271)
- Lakshminarayanan K, Padmanabhan V and Padhye J Bandwidth estimation in broadband access networks Proceedings of the 4th ACM SIGCOMM conference on Internet measurement, (314-321)
- Kapoor R, Chen L, Lao L, Gerla M and Sanadidi M (2019). CapProbe, ACM SIGCOMM Computer Communication Review, 34:4, (67-78), Online publication date: 30-Aug-2004.
- Dovrolis C, Ramanathan P and Moore D (2004). Packet-dispersion techniques and a capacity-estimation methodology, IEEE/ACM Transactions on Networking (TON), 12:6, (963-977), Online publication date: 1-Dec-2004.
- López-Ortiz A Algorithmic foundations of the internet Proceedings of the First international conference on Combinatorial and Algorithmic Aspects of Networking, (192-204)
- Lan K and Heidemann J A tool for RApid model parameterization and its applications Proceedings of the ACM SIGCOMM workshop on Models, methods and tools for reproducible network research, (76-86)
- Salamatian K and Fdida S A framework for interpreting measurement over Internet Proceedings of the ACM SIGCOMM workshop on Models, methods and tools for reproducible network research, (87-94)
- Horton J and López-Ortiz A On the number of distributed measurement points for network tomography Proceedings of the 3rd ACM SIGCOMM conference on Internet measurement, (204-209)
- Biaz S and Vaidya N Is the round-trip time correlated with the number of packets in flight? Proceedings of the 3rd ACM SIGCOMM conference on Internet measurement, (273-278)
- Bletsas A Evaluation of Kalman Filtering for Network Time Keeping Proceedings of the First IEEE International Conference on Pervasive Computing and Communications
- RäIsäNen V (2003). On end-to-end analysis of packet loss, Computer Communications, 26:14, (1693-1697), Online publication date: 1-Sep-2003.
- Saroiu S, Gummadi K and Gribble S (2003). Measuring and analyzing the characteristics of Napster and Gnutella hosts, Multimedia Systems, 9:2, (170-184), Online publication date: 1-Aug-2003.
- Loguinov D and Radha H (2002). Large-scale experimental study of Internet performance using video traffic, ACM SIGCOMM Computer Communication Review, 32:1, (7-19), Online publication date: 1-Jan-2002.
- Loguinov D and Radha H (2002). Retransmission schemes for streaming internet multimedia, ACM SIGCOMM Computer Communication Review, 32:2, (70-83), Online publication date: 1-Apr-2002.
- Barford P, Kline J, Plonka D and Ron A A signal analysis of network traffic anomalies Proceedings of the 2nd ACM SIGCOMM Workshop on Internet measurment, (71-82)
- Subramanian L, Padmanabhan V and Katz R Geographic Properties of Internet Routing Proceedings of the General Track of the annual conference on USENIX Annual Technical Conference, (243-259)
- Barford P, Bestavros A, Byers J and Crovella M On the marginal utility of network topology measurements Proceedings of the 1st ACM SIGCOMM Workshop on Internet measurement, (5-17)
- Barford P and Plonka D Characteristics of network traffic flow anomalies Proceedings of the 1st ACM SIGCOMM Workshop on Internet measurement, (69-73)
- Liu J and Crovella M Using loss pairs to discover network properties Proceedings of the 1st ACM SIGCOMM Workshop on Internet measurement, (127-138)
- Huang P, Feldmann A and Willinger W A non-instrusive, wavelet-based approach to detecting network performance problems Proceedings of the 1st ACM SIGCOMM Workshop on Internet measurement, (213-227)
- Loguinov D and Radha H Measurement study of low-bitrate internet video streaming Proceedings of the 1st ACM SIGCOMM Workshop on Internet measurement, (281-293)
- Barford P and Crovella M (2019). Critical path analysis of TCP transactions, ACM SIGCOMM Computer Communication Review, 31:2 supplement, (80-102), Online publication date: 1-Apr-2001.
- Sitaraman H, Davis J and Evans J Broadband Wireless Local Loop Evaluation via an Active TCP Connection Monitoring Methodology Proceedings of the 26th Annual IEEE Conference on Local Computer Networks
- Dooly D, Goldman S and Scott S (2001). On-line analysis of the TCP acknowledgment delay problem, Journal of the ACM (JACM), 48:2, (243-273), Online publication date: 1-Mar-2001.
- Legout A and Biersack E PLM Proceedings of the 2000 ACM SIGMETRICS international conference on Measurement and modeling of computer systems, (13-22)
- Legout A and Biersack E (2000). PLM, ACM SIGMETRICS Performance Evaluation Review, 28:1, (13-22), Online publication date: 1-Jun-2000.
- Barford P and Crovella M (2019). Critical path analysis of TCP transactions, ACM SIGCOMM Computer Communication Review, 30:4, (127-138), Online publication date: 1-Oct-2000.
- Lai K and Baker M (2019). Measuring link bandwidths using a deterministic model of packet delay, ACM SIGCOMM Computer Communication Review, 30:4, (283-294), Online publication date: 1-Oct-2000.
- Barford P and Crovella M Critical path analysis of TCP transactions Proceedings of the conference on Applications, Technologies, Architectures, and Protocols for Computer Communication, (127-138)
- Lai K and Baker M Measuring link bandwidths using a deterministic model of packet delay Proceedings of the conference on Applications, Technologies, Architectures, and Protocols for Computer Communication, (283-294)
- Ludwig R and Sklower K (2019). The Eifel retransmission timer, ACM SIGCOMM Computer Communication Review, 30:3, (17-27), Online publication date: 1-Jul-2000.
- Allman M and Paxson V On estimating end-to-end network path properties Proceedings of the conference on Applications, technologies, architectures, and protocols for computer communication, (263-274)
- Savage S, Collins A, Hoffman E, Snell J and Anderson T The end-to-end effects of Internet path selection Proceedings of the conference on Applications, technologies, architectures, and protocols for computer communication, (289-299)
- Allman M and Paxson V (2019). On estimating end-to-end network path properties, ACM SIGCOMM Computer Communication Review, 29:4, (263-274), Online publication date: 1-Oct-1999.
- Savage S, Collins A, Hoffman E, Snell J and Anderson T (2019). The end-to-end effects of Internet path selection, ACM SIGCOMM Computer Communication Review, 29:4, (289-299), Online publication date: 1-Oct-1999.
- Pan P and Schulzrinne H (2019). YESSIR, ACM SIGCOMM Computer Communication Review, 29:2, (89-101), Online publication date: 1-Apr-1999.
- Bolliger J and Gross T (1998). A Framework-Based Approach to the Development of Network-Aware Applications, IEEE Transactions on Software Engineering, 24:5, (376-390), Online publication date: 1-May-1998.
- Dooly D, Goldman S and Scott S TCP dynamic acknowledgment delay (extended abstract) Proceedings of the thirtieth annual ACM symposium on Theory of computing, (389-398)
- Luby M, Mitzenmacher M, Shokrollahi M, Spielman D and Stemann V Practical loss-resilient codes Proceedings of the twenty-ninth annual ACM symposium on Theory of computing, (150-159)
- Balakrishnan H, Stemm M, Seshan S and Katz R Analyzing stability in wide-area network performance Proceedings of the 1997 ACM SIGMETRICS international conference on Measurement and modeling of computer systems, (2-12)
- Balakrishnan H, Stemm M, Seshan S and Katz R (1997). Analyzing stability in wide-area network performance, ACM SIGMETRICS Performance Evaluation Review, 25:1, (2-12), Online publication date: 1-Jun-1997.
- Paxson V End-to-end Internet packet dynamics Proceedings of the ACM SIGCOMM '97 conference on Applications, technologies, architectures, and protocols for computer communication, (139-152)
- Paxson V Automated packet trace analysis of TCP implementations Proceedings of the ACM SIGCOMM '97 conference on Applications, technologies, architectures, and protocols for computer communication, (167-179)
- Paxson V (2019). End-to-end Internet packet dynamics, ACM SIGCOMM Computer Communication Review, 27:4, (139-152), Online publication date: 1-Oct-1997.
- Paxson V (2019). Automated packet trace analysis of TCP implementations, ACM SIGCOMM Computer Communication Review, 27:4, (167-179), Online publication date: 1-Oct-1997.
- Mathis M, Semke J, Mahdavi J and Ott T (1997). The macroscopic behavior of the TCP congestion avoidance algorithm, ACM SIGCOMM Computer Communication Review, 27:3, (67-82), Online publication date: 1-Jul-1997.
- Paxson V End-to-end routing behavior in the Internet Conference proceedings on Applications, technologies, architectures, and protocols for computer communications, (25-38)
- Paxson V (2019). End-to-end routing behavior in the Internet, ACM SIGCOMM Computer Communication Review, 26:4, (25-38), Online publication date: 1-Oct-1996.
Index Terms
- Measurements and analysis of end-to-end Internet dynamics
Recommendations
Improving accuracy in end-to-end packet loss measurement
Proceedings of the 2005 conference on Applications, technologies, architectures, and protocols for computer communicationsMeasurement and estimation of packet loss characteristics are challenging due to the relatively rare occurrence and typically short duration of packet loss episodes. While active probe tools are commonly used to measure packet loss on end-to-end paths, ...
Improving accuracy in end-to-end packet loss measurement
SIGCOMM '05: Proceedings of the 2005 conference on Applications, technologies, architectures, and protocols for computer communicationsMeasurement and estimation of packet loss characteristics are challenging due to the relatively rare occurrence and typically short duration of packet loss episodes. While active probe tools are commonly used to measure packet loss on end-to-end paths, ...