Quantitative Network Monitoring with NetQRE

ABSTRACT

In network management today, dynamic updates are required for traffic engineering and for timely response to security threats. Decisions for such updates are based on monitoring network traffic to compute numerical quantities based on a variety of network and application-level performance metrics. Today's state-of-the-art tools lack programming abstractions that capture application or session-layer semantics, and thus require network operators to specify and reason about complex state machines and interactions across layers. To address this limitation, we present the design and implementation of NetQRE, a high-level declarative toolkit that aims to simplify the specification and implementation of such quantitative network policies. NetQRE integrates regular-expression-like pattern matching at flow-level as well as application-level payloads with aggregation operations such as sum and average counts. We describe a compiler for NetQRE that automatically generates an efficient implementation with low memory footprint. Our evaluation results demonstrate that NetQRE allows natural specification of a wide range of quantitative network tasks ranging from detecting security attacks to enforcing application-layer network management policies. NetQRE results in high performance that is comparable with optimized manually-written low-level code and is significantly more efficient than alternative solutions, and can provide timely enforcement of network policies that require quantitative network monitoring.

References

1. Application Layer Packet Classifier for Linux. http://www.mcafee.com/us/products/network-security-platform.aspx.
2. CAIDA Traffic Trace. https://data.caida.org/datasets/security/ddos-20070804/.
3. McAfee Network Security Platform. http://l7-filter.sourceforge.net/.
4. OpenSketch reference code. https://github.com/USC-NSL/opensketch.
5. SIPp. http://sipp.sourceforge.net/.
6. SSL renegotiation DoS. https://www.ietf.org/mail-archive/web/tls/current/msg07553.html.
7. Anonymized 2015 Internet Traces. https://data.caida.org/datasets/passive-2015/, 2015.
8. Mohammad Al-Fares, Sivasankar Radhakrishnan, Barath Raghavan, Nelson Huang, and Amin Vahdat. Hedera: Dynamic Flow Scheduling for Data Center Networks. In NSDI, volume 10, pages 19--19, 2010.
9. Rajeev Alur, Dana Fisman, and Mukund Raghothaman. Regular Programming for Quantitative Properties of Data Streams. In 25th European Symposium on Programming. ESOP, 2016.
10. Carolyn Jane Anderson, Nate Foster, Arjun Guha, Jean-Baptiste Jeannin, Dexter Kozen, Cole Schlesinger, and David Walker. NetKAT: Semantic foundations for networks. In Proceedings of the 41st annual ACM SIGPLAN-SIGACT symposium on Principles of programming languages, pages 113--126. ACM, 2014.
11. Mina Tahmasbi Arashloo, Yaron Koral, Michael Greenberg, Jennifer Rexford, and David Walker. Snap: Stateful network-wide abstractions for packet processing. In Proceedings of the 2016 ACM SIGCOMM Conference, SIGCOMM '16, pages 29--43, New York, NY, USA, 2016. ACM.
12. Kevin Borders, Jonathan Springer, and Matthew Burnside. Chimera: A declarative language for streaming network traffic analysis. In Proceedings of the 21st USENIX Conference on Security Symposium, Security'12, pages 19--19, Berkeley, CA, USA, 2012. USENIX Association.
13. Varun Chandola, Arindam Banerjee, and Vipin Kumar. Anomaly Detection: A Survey. ACM computing surveys (CSUR), 41(3):15, 2009.
14. Sirish Chandrasekaran, Owen Cooper, Amol Deshpande, Michael J. Franklin, Joseph M. Hellerstein, Wei Hong, Sailesh Krishnamurthy, Samuel R. Madden, Fred Reiss, and Mehul A. Shah. TelegraphCQ: Continuous Dataflow Processing. In Proceedings of the 2003 ACM SIGMOD International Conference on Management of Data, SIGMOD '03, pages 668--668, New York, NY, USA, 2003. ACM.
15. Benoit Claise. Cisco systems NetFlow services export version 9. 2004.
16. Chuck Cranor, Theodore Johnson, Oliver Spataschek, and Vladislav Shkapenyuk. Gigascope: A Stream Database for Network Applications. In Proceedings of the 2003 ACM SIGMOD International Conference on Management of Data, SIGMOD '03, pages 647--651, New York, NY, USA, 2003. ACM.
17. Luca Deri. Open source VoIP traffic monitoring. In Proceedings of SANE, volume 2006, 2006.
18. Nick Duffield, Carsten Lund, and Mikkel Thorup. Estimating Flow Distributions from Sampled Flow Statistics. In Proceedings of the 2003 conference on Applications, technologies, architectures, and protocols for computer communications, pages 325--336. ACM, 2003.
19. Cristian Estan and George Varghese. New Directions in Traffic Measurement and Accounting, volume 32. ACM, 2002.
20. Seyed K. Fayaz, Yoshiaki Tobioka, Vyas Sekar, and Michael Bailey. Bohatei: Flexible and Elastic DDoS Defense. In 24th USENIX Security Symposium (USENIX Security 15), pages 817--832, Washington, D.C., August 2015. USENIX Association.
21. Nate Foster, Rob Harrison, Michael J Freedman, Christopher Monsanto, Jennifer Rexford, Alec Story, and David Walker. Frenetic: A Network Programming Language. In ACM SIGPLAN Notices, volume 46, pages 279--291. ACM, 2011.
22. Pedro Garcia-Teodoro, J Diaz-Verdejo, Gabriel Maciá-Fernández, and Enrique Vázquez. Anomaly-based Network Intrusion Detection: Techniques, Systems and Challenges. computers & security, 28(1):18--28, 2009.
23. Arpit Gupta, Rüdiger Birkner, Marco Canini, Nick Feamster, Chris Mac-Stoker, and Walter Willinger. Network Monitoring As a Streaming Analytics Problem. In Proceedings of the 15th ACM Workshop on Hot Topics in Networks, HotNets '16, pages 106--112. ACM, 2016.
24. DPDK Intel. Data Plane Development Kit. http://dpdk.org.
25. Bob Lantz, Brandon Heller, and Nick McKeown. A Network in a Laptop: Rapid Prototyping for Software-defined Networks. In Proceedings of the 9th ACM SIGCOMM Workshop on Hot Topics in Networks, Hotnets-IX, pages 19:1--19:6. ACM, 2010.
26. Boon Thau Loo, Tyson Condie, Minos Garofalakis, David E. Gay, Joseph M. Hellerstein, Petros Maniatis, Raghu Ramakrishnan, Timothy Roscoe, and Ion Stoica. Declarative Networking. CACM, 2009.
27. Konstantinos Mamouras, Mukund Raghotaman, Rajeev Alur, Zachary G. Ives, and Sanjeev Khanna. StreamQRE: Modular Specification and Efficient Evaluation of Quantitative Queries over Streaming Data. In Proceedings of the 38th ACM SIGPLAN Conference on Programming Language Design and Implementation. ACM, 2017.
28. Steve McCanne, Craig Leres, and Van Jacobson. Libpcap. http://www.tcpdump.org, 1989.
29. J Mccauley. POX: A Python-based Openflow Controller, 2014.
30. Christopher Monsanto, Joshua Reich, Nate Foster, Jennifer Rexford, David Walker, et al. Composing Software Defined Networks. In NSDI, pages 1--13, 2013.
31. Masoud Moshref, Minlan Yu, Ramesh Govindan, and Amin Vahdat. DREAM: dynamic resource allocation for software-defined measurement. In Proceedings of the 2014 ACM conference on SIGCOMM, pages 419--430. ACM, 2014.
32. Tim Nelson, Andrew D Ferguson, Michael JG Scheer, and Shriram Krishnamurthi. Tierless Programming and Reasoning for Software-Defined Networks. NSDI, Apr, 2014.
33. Vern Paxson. Bro: A System for Detecting Network Intruders in Real-time. Comput. Netw., 31(23-24):2435--2463, December 1999.
34. Martin Roesch et al. Snort: Lightweight Intrusion Detection for Networks. In LISA, volume 99, pages 229--238, 1999.
35. Vyas Sekar, Michael K Reiter, and Hui Zhang. Revisiting the case for a minimalist approach for network flow monitoring. In Proceedings of the 10th ACM SIGCOMM conference on Internet measurement, pages 328--341. ACM, 2010.
36. David Senecal. Slow DoS on the rise. https://blogs.akamai.com/2013/09/slow-dos-on-the-rise.html.
37. Robin Sommer, Matthias Vallentin, Lorenzo De Carli, and Vern Paxson. HILTI: An Abstract Execution Environment for Deep, Stateful Network Traffic Analysis. In Proceedings of the 2014 Conference on Internet Measurement Conference, IMC '14, pages 461--474, New York, NY, USA, 2014. ACM.
38. Andreas Voellmy, Junchang Wang, Y Richard Yang, Bryan Ford, and Paul Hudak. Maple: Simplifying SDN Programming using Algorithmic Policies. In Proceedings of the ACM SIGCOMM 2013 conference on SIGCOMM, pages 87--98. ACM, 2013.
39. Mea Wang, Baochun Li, and Zongpeng Li. sFlow: Towards resource-efficient and agile service federation in service overlay networks. In Distributed Computing Systems, 2004. Proceedings. 24th International Conference on, pages 628--635. IEEE, 2004.
40. Minlan Yu, Lavanya Jose, and Rui Miao. Software Defined Traffic Measurement with OpenSketch. In NSDI, volume 13, pages 29--42, 2013.
41. Lihua Yuan, Chen-Nee Chuah, and Prasant Mohapatra. ProgME: Towards Programmable Network Measurement. IEEE/ACM Transactions on Networking (TON), 19(1):115--128, 2011.

Index Terms

• Networks

  • Network services

    • Network monitoring

    • Programmable networks