Abstract
Many recent caching systems aim to improve miss ratios, but there is no good sense among practitioners of how much further miss ratios can be improved. In other words, should the systems community continue working on this problem? Currently, there is no principled answer to this question. In practice, object sizes often vary by several orders of magnitude, where computing the optimal miss ratio (OPT) is known to be NP-hard. The few known results on caching with variable object sizes provide very weak bounds and are impractical to compute on traces of realistic length. We propose a new method to compute upper and lower bounds on OPT. Our key insight is to represent caching as a min-cost flow problem, hence we call our method the flow-based offline optimal (FOO). We prove that, under simple independence assumptions, FOO's bounds become tight as the number of objects goes to infinity. Indeed, FOO's error over 10M requests of production CDN and storage traces is negligible: at most 0.3%. FOO thus reveals, for the first time, the limits of caching with variable object sizes. While FOO is very accurate, it is computationally impractical on traces with hundreds of millions of requests. We therefore extend FOO to obtain more efficient bounds on OPT, which we call practical flow-based offline optimal (PFOO). We evaluate PFOO on several full production traces and use it to compare OPT to prior online policies. This analysis shows that current caching systems are in fact still far from optimal, suffering 11--43% more cache misses than OPT, whereas the best prior offline bounds suggest that there is essentially no room for improvement.
- Marc Abrams, C. R. Standridge, Ghaleb Abdulla, S. Williams, and Edward A. Fox . 1995. Caching Proxies: Limitations and Potentials. Technical Report. Virginia Polytechnic Institute & State University Blacksburgh, VA. Google ScholarDigital Library
- Alfred V Aho, Peter J Denning, and Jeffrey D Ullman . 1971. Principles of optimal page replacement. J. ACM Vol. 18, 1 (1971), 80--93. Google ScholarDigital Library
- Ravindra K Ahuja, Thomas L Magnanti, and James B Orlin . 1993. Network flows: theory, algorithms, and applications. Prentice hall. Google ScholarDigital Library
- Susanne Albers, Sanjeev Arora, and Sanjeev Khanna . 1999. Page replacement for general caching problems. In SODA. 31--40. Google ScholarDigital Library
- Emmanuelle Anceaume, Yann Busnel, and Bruno Sericola . 2015. New results on a generalized coupon collector problem using Markov chains. Journal of Applied Probability Vol. 52, 2 (2015), 405--418.Google ScholarCross Ref
- Omri Bahat and Armand M Makowski . 2003. Optimal replacement policies for nonuniform cache objects with optional eviction IEEE INFOCOM. 427--437.Google Scholar
- Amotz Bar-Noy, Reuven Bar-Yehuda, Ari Freund, Joseph Naor, and Baruch Schieber . 2001. A unified approach to approximating resource allocation and scheduling. J. ACM Vol. 48, 5 (2001), 1069--1090. Google ScholarDigital Library
- Ruben Becker, Maximilian Fickert, and Andreas Karrenbauer . {n. d.}. A Novel Dual Ascent Algorithm for Solving the Min-Cost Flow Problem. Chapter 12, 151--159.Google Scholar
- Ruben Becker and Andreas Karrenbauer . 2013. A Combinatorial O(m 3/2)-time Algorithm for the Min-Cost Flow Problem. arXiv preprint arXiv:1312.3905 (2013).Google Scholar
- Nathan Beckmann, Haoxian Chen, and Asaf Cidon . 2018. LHD: Improving Hit Rate by Maximizing Hit Density. USENIX NSDI. 1--14.Google Scholar
- Nathan Beckmann and Daniel Sanchez . 2015. Talus: A Simple Way to Remove Cliffs in Cache Performance IEEE HPCA. 64--75.Google Scholar
- Nathan Beckmann and Daniel Sanchez . 2016. Modeling cache behavior beyond LRU. In IEEE HPCA. 225--236.Google Scholar
- L. A. Belady . 1996. A study of replacement algorithms for a virtual-storage computer. IBM Systems journal Vol. 5 (1996), 78--101. Google ScholarDigital Library
- Daniel S. Berger, Philipp Gland, Sahil Singla, and Florin Ciucu . 2014. Exact analysis of TTL cache networks. Perform. Eval. Vol. 79 (2014), 2 -- 23. Special Issue: Performance 2014. Google ScholarDigital Library
- Daniel S Berger, Sebastian Henningsen, Florin Ciucu, and Jens B Schmitt . 2015. Maximizing cache hit ratios by variance reduction. ACM SIGMETRICS Performance Evaluation Review, Vol. 43, 2 (2015), 57--59. Google ScholarDigital Library
- Daniel S. Berger, Ramesh Sitaraman, and Mor Harchol-Balter . 2017. AdaptSize: Orchestrating the Hot Object Memory Cache in a CDN USENIX NSDI. 483--498. Google ScholarDigital Library
- Aaron Blankstein, Siddhartha Sen, and Michael J Freedman . 2017. Hyperbolic Caching: Flexible Caching for Web Applications USENIX ATC. 499--511. Google ScholarDigital Library
- Béla Bollobás and Graham Brightwell . 1992. The height of a random partial order: concentration of measure. The Annals of Applied Probability (1992), 1009--1018.Google Scholar
- Lee Breslau, Pei Cao, Li Fan, Graham Phillips, and Scott Shenker . 1999. Web caching and Zipf-like distributions: Evidence and implications IEEE INFOCOM. 126--134.Google Scholar
- PJ Burville and JFC Kingman . 1973. On a model for storage and search. Journal of Applied Probability (1973), 697--701.Google Scholar
- Hao Che, Ye Tung, and Zhijun Wang . 2002. Hierarchical web caching systems: Modeling, design and experimental results. IEEE JSAC Vol. 20 (2002), 1305--1314. Google ScholarDigital Library
- Ludmila Cherkasova . 1998. Improving WWW proxies performance with greedy-dual-size-frequency caching policy. Technical Report. Hewlett-Packard Laboratories.Google Scholar
- Marek Chrobak, Gerhard J Woeginger, Kazuhisa Makino, and Haifeng Xu . 2012. Caching is hard-even in the fault model. Algorithmica Vol. 63 (2012), 781--794. Google ScholarDigital Library
- Asaf Cidon, Assaf Eisenman, Mohammad Alizadeh, and Sachin Katti . 2016. Cliffhanger: Scaling Performance Cliffs in Web Memory Caches USENIX NSDI. 379--392. Google ScholarDigital Library
- Edward Grady Coffman and Peter J Denning . 1973. Operating systems theory. Prentice-Hall. Google ScholarDigital Library
- Edward Grady Coffman and Predrag Jelenković . 1999. Performance of the move-to-front algorithm with Markov-modulated request sequences. Operations Research Letters Vol. 25 (1999), 109--118. Google ScholarDigital Library
- Leonardo Dagum and Ramesh Menon . 1998. OpenMP: an industry standard API for shared-memory programming. IEEE Computational Science and Engineering, Vol. 5, 1 (1998), 46--55. Google ScholarDigital Library
- Samuel I Daitch and Daniel A Spielman . 2008. Faster approximate lossy generalized flow via interior point algorithms ACM STOC. 451--460. Google ScholarDigital Library
- Asit Dan and Don Towsley . 1990. An Approximate Analysis of the LRU and FIFO Buffer Replacement Schemes ACM SIGMETRICS. 143--152. Google ScholarDigital Library
- Wayne W Daniel et almbox. . 1978. Applied nonparametric statistics. Houghton Mifflin.Google Scholar
- Robert P Dobrow and James Allen Fill . 1995. The move-to-front rule for self-organizing lists with Markov dependent requests. Discrete Probability and Algorithms. Springer, 57--80.Google Scholar
- Aristides V Doumas and Vassilis G Papanicolaou . 2012. The coupon collector's problem revisited: asymptotics of the variance. Advances in Applied Probability Vol. 44, 1 (2012), 166--195.Google ScholarCross Ref
- Ding-Zhu Du and Panos M Pardalos . 2013. Handbook of combinatorial optimization (bibinfoedition2 ed.). Springer.Google Scholar
- Gil Einziger and Roy Friedman . 2014. Tinylfu: A highly efficient cache admission policy IEEE Euromicro PDP. 146--153. Google ScholarDigital Library
- William Feller . 2008. An introduction to probability theory and its applications. Vol. Vol. 2. John Wiley & Sons.Google Scholar
- Andrés Ferragut, Ismael Rodr'ıguez, and Fernando Paganini . 2016. Optimizing TTL caches under heavy-tailed demands. ACM SIGMETRICS. 101--112. Google ScholarDigital Library
- James Allen Fill and Lars Holst . 1996. On the distribution of search cost for the move-to-front rule. Random Structures & Algorithms Vol. 8 (1996), 179--186. Google ScholarDigital Library
- Philippe Flajolet, Daniele Gardy, and Loÿs Thimonier . 1992. Birthday paradox, coupon collectors, caching algorithms and self-organizing search. Discrete Applied Mathematics Vol. 39 (1992), 207--229. Google ScholarDigital Library
- Lukávs Folwarcznỳ and Jivr'ı Sgall . 2017. General caching is hard: Even with small pages. Algorithmica, Vol. 79, 2 (2017), 319--339. Google ScholarDigital Library
- Christine Fricker, Philippe Robert, and James Roberts . 2012. A versatile and accurate approximation for LRU cache performance ITC. 8--16.Google Scholar
- Massimo Gallo, Bruno Kauffmann, Luca Muscariello, Alain Simonian, and Christian Tanguy . 2012. Performance evaluation of the random replacement policy for networks of caches ACM SIGMETRICS/ PERFORMANCE. 395--396. Google ScholarDigital Library
- Nicolas Gast and Benny Van Houdt . 2015. Transient and steady-state regime of a family of list-based cache replacement algorithms ACM SIGMETRICS. 123--136. Google ScholarDigital Library
- Erol Gelenbe . 1973. A unified approach to the evaluation of a class of replacement algorithms. IEEE Trans. Comput. Vol. 100 (1973), 611--618. Google ScholarDigital Library
- Binny S Gill . 2008. On multi-level exclusive caching: offline optimality and why promotions are better than demotions. In USENIX FAST. 4--18. Google ScholarDigital Library
- WJ Hendricks . 1972. The stationary distribution of an interesting Markov chain. Journal of Applied Probability (1972), 231--233.Google Scholar
- Peter Hillmann, Tobias Uhlig, Gabi Dreo Rodosek, and Oliver Rose . 2016. Simulation and optimization of content delivery networks considering user profiles and preferences of internet service providers IEEE Winter Simulation Conference. 3143--3154. Google ScholarDigital Library
- Wassily Hoeffding . 1948. A non-parametric test of independence. The annals of mathematical statistics (1948), 546--557.Google Scholar
- Qi Huang, Ken Birman, Robbert van Renesse, Wyatt Lloyd, Sanjeev Kumar, and Harry C Li . 2013. An analysis of Facebook photo caching. In ACM SOSP. 167--181. Google ScholarDigital Library
- Stratis Ioannidis and Edmund Yeh . 2016. Adaptive caching networks with optimality guarantees ACM SIGMETRICS. 113--124. Google ScholarDigital Library
- Sandy Irani . 1997. Page replacement with multi-size pages and applications to web caching ACM STOC. 701--710. Google ScholarDigital Library
- Akanksha Jain and Calvin Lin . 2016. Back to the future: leveraging Belady's algorithm for improved cache replacement ACM/IEEE ISCA. 78--89. Google ScholarDigital Library
- Predrag R Jelenković . 1999. Asymptotic approximation of the move-to-front search cost distribution and least-recently used caching fault probabilities. The Annals of Applied Probability Vol. 9 (1999), 430--464.Google ScholarCross Ref
- Predrag R Jelenković and Ana Radovanović . 2004. Least-recently-used caching with dependent requests. Theoretical computer science Vol. 326 (2004), 293--327. Google ScholarDigital Library
- Predrag R Jelenkovic and Ana Radovanovic . 2004. Optimizing LRU caching for variable document sizes. Combinatorics, Probability and Computing Vol. 13 (2004), 627--643. Google ScholarDigital Library
- Mahesh Kallahalla and Peter J Varman . 2002. PC-OPT: optimal offline prefetching and caching for parallel I/O systems. IEEE Trans. Comput. Vol. 51, 11 (2002), 1333--1344. Google ScholarDigital Library
- Brian Karrer and Mark EJ Newman . 2009. Random graph models for directed acyclic networks. Physical Review E, Vol. 80, 4 (2009), 046110.Google ScholarCross Ref
- Richard E. Kessler, Mark D Hill, and David A Wood . 1994. A comparison of trace-sampling techniques for multi-megabyte caches. IEEE Trans. Comput. Vol. 43, 6 (1994), 664--675. Google ScholarDigital Library
- W. Frank King . 1971. Analysis of Demand Paging Algorithms. In IFIP Congress (1). 485--490.Google Scholar
- Christos Koufogiannakis and Neal E Young . 2014. A nearly linear-time PTAS for explicit fractional packing and covering linear programs. Algorithmica, Vol. 70, 4 (2014), 648--674. Google ScholarDigital Library
- Conglong Li and Alan L Cox . 2015. GD-Wheel: a cost-aware replacement policy for key-value stores EUROSYS. 1--15. Google ScholarDigital Library
- Suoheng Li, Jie Xu, Mihaela van der Schaar, and Weiping Li . 2016. Popularity-driven content caching. In IEEE INFOCOM. 1--9.Google Scholar
- Bruce M Maggs and Ramesh K Sitaraman . 2015. Algorithmic nuggets in content delivery. ACM SIGCOMM CCR Vol. 45 (2015), 52--66. Google ScholarDigital Library
- Valentina Martina, Michele Garetto, and Emilio Leonardi . 2014. A unified approach to the performance analysis of caching systems. IEEE INFOCOM. 12--24.Google Scholar
- Richard L. Mattson, Jan Gecsei, Donald R. Slutz, and Irving L. Traiger . 1970. Evaluation techniques for storage hierarchies.. In IBM Systems journal, Vol. Vol. 9. 78--117. Google ScholarDigital Library
- John McCabe . 1965. On serial files with relocatable records. Operations Research Vol. 13 (1965), 609--618. Google ScholarDigital Library
- Pierre Michaud . 2016. Some mathematical facts about optimal cache replacement. ACM Transactions on Architecture and Code Optimization (TACO), Vol. 13, 4 (2016), 50. Google ScholarDigital Library
- John Moriarty and Peter Neal . 2008. The Generalized Coupon Collector Problem. Journal of Applied Probability Vol. 45 (2008), 621--29.Google ScholarCross Ref
- Giovanni Neglia, Damiano Carra, and Pietro Michiardi . 2018. Cache policies for linear utility maximization. IEEE/ACM Transactions on Networking Vol. 26, 1 (2018), 302--313. Google ScholarDigital Library
- Rajesh Nishtala, Hans Fugal, Steven Grimm, Marc Kwiatkowski, Herman Lee, Harry C Li, Ryan McElroy, Mike Paleczny, Daniel Peek, Paul Saab, et almbox. . 2013. Scaling memcache at facebook. In USENIX NSDI. 385--398. Google ScholarDigital Library
- Egerváry Research Group on Combinatorial Optimization . 2015. COIN-OR::LEMON Library. (2015). Available at http://lemon.cs.elte.hu/trac/lemon, accessed 10/21/17.Google Scholar
- Elizabeth J O'Neil, Patrick E O'Neil, and Gerhard Weikum . 1993. The LRU-K page replacement algorithm for database disk buffering. ACM SIGMOD, Vol. 22, 2 (1993), 297--306. Google ScholarDigital Library
- Elizabeth J O'Neil, Patrick E O'Neil, and Gerhard Weikum . 1999. An optimality proof of the LRU-K page replacement algorithm. JACM Vol. 46 (1999), 92--112. Google ScholarDigital Library
- Antonis Panagakis, Athanasios Vaios, and Ioannis Stavrakakis . 2008. Approximate analysis of LRU in the case of short term correlations. Computer Networks Vol. 52 (2008), 1142--1152. Google ScholarDigital Library
- Konstantinos Psounis and Balaji Prabhakar . 2001. A randomized web-cache replacement scheme. In INFOCOM 2001. Twentieth Annual Joint Conference of the IEEE Computer and Communications Societies. Proceedings. IEEE, Vol. Vol. 3. IEEE, 1407--1415.Google ScholarCross Ref
- Konstantinos Psounis, An Zhu, Balaji Prabhakar, and Rajeev Motwani . 2004. Modeling correlations in web traces and implications for designing replacement policies. Computer Networks Vol. 45 (2004), 379--398. Google ScholarDigital Library
- Eliane R Rodrigues . 1995. The performance of the move-to-front scheme under some particular forms of Markov requests. Journal of applied probability Vol. 32, 4 (1995), 1089--1102.Google ScholarCross Ref
- Samta Shukla and Alhussein A Abouzeid . 2017. Optimal Device-Aware Caching. IEEE Transactions on Mobile Computing Vol. 16, 7 (2017), 1994--2007.Google ScholarDigital Library
- Ramesh K. Sitaraman, Mangesh Kasbekar, Woody Lichtenstein, and Manish Jain . 2014. Overlay networks: An Akamai perspective. Advanced Content Delivery, Streaming, and Cloud Services. John Wiley & Sons.Google Scholar
- Storage Networking Industry Association (SNIA) . 2008. MSR Cambridge Traces. http://iotta.snia.org/traces/388. (2008).Google Scholar
- David Starobinski and David Tse . 2001. Probabilistic methods for web caching. Perform. Eval. Vol. 46 (2001), 125--137. Google ScholarDigital Library
- O. Tange . 2011. GNU Parallel - The Command-Line Power Tool. ;login: The USENIX Magazine Vol. 36, 1 (Feb . 2011), 42--47.Google Scholar
- Naoki Tsukada, Ryo Hirade, and Naoto Miyoshi . 2012. Fluid limit analysis of FIFO and RR caching for independent reference model. Perform. Eval. Vol. 69 (Sept. . 2012), 403--412. Google ScholarDigital Library
- Vijay V Vazirani . 2013. Approximation algorithms. Springer Science & Business Media. Google ScholarDigital Library
- Carl Waldspurger, Trausti Saemundsson, Irfan Ahmad, and Nohhyun Park . 2017. Cache Modeling and Optimization using Miniature Simulations USENIX. 487--498). Google ScholarDigital Library
- Weiyu Xu and A. Kevin Tang . 2011. A Generalized Coupon Collector Problem. Journal of Applied Probability Vol. 48, 4 (2011), 1081--1094.Google ScholarCross Ref
- Neal E Young . 2000. Online paging against adversarially biased random inputs. Journal of Algorithms Vol. 37 (2000), 218--235. Google ScholarDigital Library
Index Terms
- Practical Bounds on Optimal Caching with Variable Object Sizes
Recommendations
Practical Bounds on Optimal Caching with Variable Object Sizes
SIGMETRICS '18Many recent caching systems aim to improve miss ratios, but there is no good sense among practitioners of how much further miss ratios can be improved. In other words, should the systems community continue working on this problem? Currently, there is no ...
Practical Bounds on Optimal Caching with Variable Object Sizes
SIGMETRICS '18: Abstracts of the 2018 ACM International Conference on Measurement and Modeling of Computer SystemsMany recent caching systems aim to improve miss ratios, but there is no good sense among practitioners of how much further miss ratios can be improved. In other words, should the systems community continue working on this problem? Currently, there is no ...
Pacman: program-assisted cache management
ISMM '13: Proceedings of the 2013 international symposium on memory managementAs caches become larger and shared by an increasing number of cores, cache management is becoming more important. This paper explores collaborative caching, which uses software hints to influence hardware caching. Recent studies have shown that such ...
Comments