Sa’di Altamimi
Shervin Shirmohammadi
Skip Abstract Section
Abstract
To design an optimal adaptive video streaming method, video service providers need to consider both the efficiency and the fairness of the Quality of Experience (QoE) of their users. In Reference [8], we proposed a server-side QoE-fair rate adaptation method that considers both efficiency and fairness of the QoE. The server uses Reinforcement Learning (RL) to select a bitrate for each client sharing the same bottleneck link to the server in a way that achieves fairness among concurrent DASH clients and imposes that bitrate by dynamically modifying the client’s Media Presentation Description (MPD) file. In this article, we extend that work to minimize the number of actions the server needs to take to keep the system in its equilibrium state. By incorporating a Recurrent Neural Network, specifically an LSTM model, we modify the server’s training algorithm to achieve improvements in both the quality and the quantity of actions the server takes to guide the client. Performance evaluation of the modified algorithm for clients running both homogeneous and heterogeneous adaptation algorithms showed that the number of server actions dropped by 14% and 22%, respectively, while QoE-fairness improved by at least 6% and 10%, respectively.
- 2012. ITU-T Recommendation G.1070 Opinion Model for Video Applications.Google Scholar
- 2014. ISO/IEC 23009-1, Information Technology - Dynamic Adaptive Streaming Over HTTP (DASH) - Part 1: Media Presentation Description and Segment Formats.Google Scholar
- 2020. Global Internet Phenomena Report. Sandvine, September 2019. Retrieved March 29, 2020 from https://www.sandvine.com/press-releases/sandvine-releases-2019-global-internet-phenomena-report.Google Scholar
- 2020. The Mobile Internet Phenomena Report. Sandvine. Retrieved March 29, 2020 from https://www.sandvine.com/press-releases/sandvine-releases-2020-mobile-internet-phenomena-report-youtube-is-over-25-of-all-mobile-traffic.Google Scholar
- Saamer Akhshabi, Lakshmi Anantakrishnan, Ali C. Begen, and Constantine Dovrolis. 2012. What happens when HTTP adaptive streaming players compete for bandwidth? In Proceedings of the 22nd International Workshop on Network and Operating System Support for Digital Audio and Video (NOSSDAV’12). ACM, New York, NY, 9--14. DOI:https://doi.org/10.1145/2229087.2229092Google ScholarDigital Library
- Saamer Akhshabi, Lakshmi Anantakrishnan, Constantine Dovrolis, and Ali Begen. 2013. Server-based traffic shaping for stabilizing oscillating adaptive streaming players. In Proceeding of the 23rd ACM Workshop on Network and Operating Systems Support for Digital Audio and Video (NOSSDAV’13). ACM, 19--24.Google ScholarDigital Library
- Zahaib Akhtar, Yun Seong Nam, Ramesh Govindan, Sanjay Rao, Jessica Chen, Ethan Katz-Bassett, Bruno Ribeiro, Jibin Zhan, and Hui Zhang. 2018. Oboe: Auto-tuning video ABR algorithms to network conditions. In Proceedings of the 2018 Conference of the ACM Special Interest Group on Data Communication (SIGCOMM’18). CM, New York, NY, 44--58. DOI:https://doi.org/10.1145/3230543.3230558Google ScholarDigital Library
- Sa’di Altamimi and Shervin Shirmohammadi. 2019. Client-server cooperative and fair DASH video streaming. In Proceedings of the 29th ACM Workshop on Network and Operating Systems Support for Digital Audio and Video (NOSSDAV’19). ACM, New York, NY, USA, 1--6. DOI:https://doi.org/10.1145/3304112.3325604Google ScholarDigital Library
- Abdelhak Bentaleb, Ali C. Begen, and Roger Zimmermann. 2018. ORL-SDN: Online reinforcement learning for SDN-enabled HTTP adaptive streaming. ACM Trans. Multimedia Comput. Commun. Appl. 14, 3, Article 71 (Aug. 2018), 28 pages. DOI:https://doi.org/10.1145/3219752Google ScholarDigital Library
- Tyler Chesebro and Aleks Kamko. 2016. Learning Atari: An Exploration of the A3C Reinforcement Learning Method. Available: https://bcourses.berkeley.edu/files/70573736/download?download_frd=1.Google Scholar
- Federal Communications Commission. [n.d.]. Raw Data - Measuring Broadband America. Retrieved from https://www.fcc.gov/reports-research/reports.Google Scholar
- M. Gadaleta, F. Chiariotti, M. Rossi, and A. Zanella. 2017. D-DASH: A deep Q-learning framework for DASH video streaming. IEEE Trans. Cogn. Commun. Netw. 3, 4 (2017), 703--718.Google ScholarCross Ref
- Mahdi Hemmati. 2017. New Bandwidth Allocation Methods to Provide Quality-of-Experience Fairness for Video Streaming Services. Ph.D. Dissertation. University of Ottawa, School of Electrical Engineering 8 Computer Science.Google Scholar
- Mahdi Hemmati, Abdulsalam Yassine, and Shervin Shirmohammadi. 2015. A Dec-POMDP model for congestion avoidance and fair allocation of network bandwidth in rate-adaptive video streaming. In Proceedings of the 2015 IEEE Symposium Series on Computational Intelligence. 1182--1189. DOI:https://doi.org/10.1109/SSCI.2015.170Google ScholarCross Ref
- T. Hoßfeld, L. Skorin-Kapov, P. E. Heegaard, and M. Varela. 2017. Definition of QoE fairness in shared systems. IEEE Commun. Lett. 21, 1 (2017), 184--187. https://ieeexplore.ieee.org/document/7588099.Google ScholarCross Ref
- T. Huang, X. Yao, C. Wu, R. Zhang, Z. Pang, and L. Sun. 2019. Tiyuntsong: A self-play reinforcement learning approach for ABR video streaming. In Proceedings of the 2019 IEEE International Conference on Multimedia and Expo (ICME’19). 1678--1683.Google Scholar
- Tianchi Huang, Chao Zhou, Rui-Xiao Zhang, Chenglei Wu, Xin Yao, and Lifeng Sun. 2019. Comyco: Quality-aware adaptive video streaming via imitation learning. In Proceedings of the 27th ACM International Conference on Multimedia (MM’19). ACM, New York, NY, 429--437. DOI:https://doi.org/10.1145/3343031.3351014Google ScholarDigital Library
- Te-Yuan Huang, Nikhil Handigol, Brandon Heller, Nick McKeown, and Ramesh Johari. 2012. Confused, timid, and unstable: Picking a video streaming rate is hard. In Proceedings of the 2012 Internet Measurement Conference (IMC’12). ACM, New York, NY, 225--238. DOI:https://doi.org/10.1145/2398776.2398800Google ScholarDigital Library
- R. Jain, D.-M. Chiu, W. R. Hawe, T. Zinner, T. Hoßfeld, and P. Tran-Gia. 1984. A quantitative measure of fairness and discrimination for resource allocation in shared computer system. Eastern Res. Lab. (1984), Hudson, MA.Google Scholar
- Junchen Jiang, Vyas Sekar, and Hui Zhang. 2014. Improving fairness, efficiency, and stability in HTTP-based adaptive video streaming with festive. IEEE/ACM Trans. Netw. 22, 1 (Feb. 2014), 326--340. DOI:https://doi.org/10.1109/TNET.2013.2291681Google ScholarDigital Library
- Vijay Konda and John Tsitsiklis. 2003. On actor-critic algorithms. SIAM J. Contr. Optim. 42, 4 (2003), 1143--1166.Google ScholarDigital Library
- Stefan Lederer, Christopher Muller, and Christian Timmerer. 2012. Dynamic adaptive streaming over HTTP dataset. In Proceedings of the 3rd Multimedia Systems Conference (MMSys’12). ACM, New York, NY, 89--94. DOI:https://doi.org/10.1145/2155555.2155570Google ScholarDigital Library
- De Cicco Luca, Saverio Mascolo, and Vittorio Palmisano. 2011. Feedback control for adaptive live video streaming. In Proceedings of the 2nd Annual ACM Conference on Multimedia Systems (MMSys’11). ACM, 145--156.Google Scholar
- Hongzi Mao, Ravi Netravali, and Mohammad Alizadeh. 2017. Neural adaptive video streaming with pensieve. Proceedings of the ACM Special Interest Group on Data Communication (SIGCOMM’17).Google ScholarDigital Library
- Volodymyr Mnih, Koray Kavukcuoglu, David Silver, et al. 2015. Human-level control through deep reinforcement learning. Nature 25, 2, 529--533. http://dx.doi.org/10.1038/518486aGoogle ScholarCross Ref
- A. Nadembega, A. Hafid, and T. Taleb. 2014. An integrated predictive mobile-oriented bandwidth-reservation framework to support mobile multimedia streaming. IEEE Trans. Wireless Commun. 13, 12 (2014), 6863--6875.Google ScholarCross Ref
- Vikram Nathan, Vibhaalakshmi Sivaraman, Ravichandra Addanki, Mehrdad Khani, Prateesh Goyal, and Mohammad Alizadeh. 2019. End-to-end transport for video QoE fairness. In Proceedings of the ACM Special Interest Group on Data Communication (SIGCOMM’19). ACM, New York, NY, 408--423. DOI:https://doi.org/10.1145/3341302.3342077Google ScholarDigital Library
- Haakon Riiser, Paul Vigmostad, Carsten Griwodz, and Halvorsen Pal. 2013. Commute path bandwidth traces from 3G networks: Analysis and applications. In Proceedings of the 4th ACM Multimedia Systems Conference (MMSys’13). ACM, New York, NY, 114--118. DOI:https://doi.org/10.1145/2483977.2483991Google ScholarDigital Library
- J. Padhye S. Floyd, M. Handley and J. Widmer. 2008. TCP Friendly Rate Control (TFRC): Protocol Specification (Proposed Standard). Retrieved from https://tools.ietf.org/html/rfc5348.Google Scholar
- M. Seufert, S. Egger, M. Slanina, T. Zinner, T. Hoßfeld, and P. Tran-Gia. 2015. A survey on quality of experience of HTTP adaptive streaming. IEEE Commun. Surv. Tutor. 17, 1 (2015), 469--492.Google ScholarDigital Library
- M. Seufert, N. Wehner, and P. Casas. 2019. A fair share for all: TCP-inspired adaptation logic for QoE fairness among heterogeneous HTTP adaptive video streaming clients. IEEE Trans. Netw. Serv. Manag. 16, 2 (2019), 475--488.Google ScholarCross Ref
- Ashkan Sobhani, Abdulsalam Yassine, and Shervin Shirmohammadi. 2017. A video bitrate adaptation and prediction mechanism for HTTP adaptive streaming. ACM Trans. Multimedia Comput. Commun. Appl. 13, 2, (Mar. 2017), 25 pages. DOI:https://doi.org/10.1145/3052822Google ScholarDigital Library
- Kevin Spiteri, Rahul Urgaonkar, and Ramesh K. Sitaraman. 2016. BOLA: Near-optimal bitrate adaptation for online videos. In Proceedings of the 35th Annual IEEE International Conference on Computer Communications (INFOCOM’16). 1--9. DOI:https://doi.org/10.1109/INFOCOM.2016.7524428Google Scholar
- Richard Sutton and Andrew Barto. 1998. Reinforcement Learning: An Introduction. MIT Press.Google ScholarDigital Library
- The PyTorch team. 2020. PyTorch Neural Network Framework. Retrieved from https://pytorch.org/docs/stable/nn.html#lstm.Google Scholar
- Xiaoqi Yin, Abhishek Jindal, Vyas Sekar, and Bruno Sinopoli. 2015. A control-theoretic approach for dynamic adaptive video streaming over HTTP. In Proceedings of the 2015 ACM Conference on Special Interest Group on Data Communication (SIGCOMM’15). ACM, New York, NY, 325--338. DOI:https://doi.org/10.1145/2785956.2787486Google ScholarDigital Library
Index Terms
- QoE-Fair DASH Video Streaming Using Server-side Reinforcement Learning
Recommendations
Client-server cooperative and fair DASH video streaming
NOSSDAV '19: Proceedings of the 29th ACM Workshop on Network and Operating Systems Support for Digital Audio and VideoAdaptive video streaming over HTTP, such as the MPEG-DASH standard, is now widely used by video service provides to stream their videos to users. But DASH and similar methods are known to suffer from two practical challenges: on the one hand, clients ...
DQ-DASH: A Queuing Theory Approach to Distributed Adaptive Video Streaming
The significant popularity of HTTP adaptive video streaming (HAS), such as Dynamic Adaptive Streaming over HTTP (DASH), over the Internet has led to a stark increase in user expectations in terms of video quality and delivery robustness. This situation ...
Reinforcement learning-based rate adaptation in dynamic video streaming
AbstractVideo streaming stands out as the most significant traffic type consumed by mobile devices. This increased demand has been a major driver for research on bitrate adaptation algorithms. Bitrate adaptation ensures high user-perceived quality, which, ...
Comments