Rui-Xiao Zhang
Skip Abstract Section
Abstract
Scheduling viewers effectively among different Content Delivery Network (CDN) providers is challenging owing to the extreme diversity in the crowdsourced live streaming (CLS) scenarios. Abundant algorithms have been proposed in recent years, which, however, suffer from a critical limitation: Due to their inaccurate feature engineering or naive rules, they cannot optimally schedule viewers. To address this concern, we put forward LTS (Learn to Schedule), a novel scheduling algorithm that can adapt to the dynamics from both viewer traffics and CDN performance. In detail, we first propose LTS-RL, an approach that schedules CLS viewers based on deep reinforcement learning (DRL). Since LTS-RL is trained in an end-to-end way, it can automatically learn scheduling algorithms without any pre-programmed models or assumptions about the environment dynamics. At the same time, to practically deploy LTS-RL, we then use the decision tree and imitation learning to convert LTS-RL into a more light-weighted and interpretable model, which is denoted as Fast-LTS. After the extensive evaluation of the real data from a leading CLS platform in China, we demonstrate that our proposed model (both LTS-RL and Fast-LTS) can improve the average quality of experience (QoE) over state-of-the-art approaches by 8.71--15.63%. At the same time, we also demonstrate that Fast-LTS can faithfully convert the complicated LTS-RL with slight performance degradation (< 2%), while significantly reducing the decision time (×7--10).
- Vijay Kumar Adhikari et al. 2012. Unreeling netflix: Understanding and improving multi-cdn movie delivery. In Proceedings of the IEEE International Conference on Computer Communications (INFOCOM’12). IEEE, 1620--1628.Google Scholar
- Vijay Kumar Adhikari et al. 2012. A tale of three CDNs: An active measurement study of Hulu and its CDNs. In Proceedings of the 2012 IEEE Conference on Computer Communications Workshops (INFOCOM WKSHPS’12). IEEE, 7--12.Google Scholar
- Alexandre Alahi, Kratarth Goel, Vignesh Ramanathan, Alexandre Robicquet, Li Feifei, and Silvio Savarese. 2016. Social LSTM: Human trajectory prediction in crowded spaces. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 961--971.Google ScholarCross Ref
- Hendrik Blockeel and Luc De Raedt. 1998. Top-down induction of first-order logical decision trees. Artif. Intell. 101, 1--2 (1998), 285--297.Google ScholarCross Ref
- Leo Breiman. 2017. Classification and Regression Trees. Routledge.Google Scholar
- Arnaud Dethise, Marco Canini, and Srikanth Kandula. 2019. Cracking open the black box: What observations can tell us about reinforcement learning agents. In Proceedings of the 2019 Workshop on Network Meets AI 8 ML. ACM, 29--36.Google ScholarDigital Library
- Florin Dobrian and et al. 2011. Understanding the impact of video quality on user engagement. In ACM SIGCOMM Computer Communication Review, Vol. 41. ACM, 362--373.Google Scholar
- Mnih et al. 2016. Asynchronous methods for deep reinforcement learning. In Proceedings of the International Conference on Machine Learning. 1928--1937.Google Scholar
- Chen Fei et al. 2015. Cloud-assisted live streaming for crowdsourced multimedia content. IEEE Trans. Multimedia 17, 9 (2015), 1471--1483.Google ScholarDigital Library
- Rui Fu, Zuo Zhang, and L. Li. 2016. Using LSTM and GRU neural network methods for traffic flow prediction. In 2016 31st Youth Academic Annual Conference of Chinese Association of Automation (YAC’16). IEEE, 324–328.Google Scholar
- Aurélien Garivier and Eric Moulines. 2008. On upper-confidence bound policies for non-stationary bandit problems. arXiv preprint arXiv:0805.3415 (2008).Google Scholar
- Wenbo Guo, Dongliang Mu, Jun Xu, Purui Su, Gang Wang, and Xinyu Xing. 2018. Lemna: Explaining deep learning based security applications. In Proceedings of the 2018 ACM SIGSAC Conference on Computer and Communications Security. ACM, 364--379.Google ScholarDigital Library
- Tianchi Huang, Rui-Xiao Zhang, Chao Zhou, and Lifeng Sun. 2018. QARC: Video quality aware rate control for real-time video streaming based on deep reinforcement learning. arXiv preprint arXiv:1805.02482 (2018).Google Scholar
- Tianchi Huang, Chao Zhou, Rui-Xiao Zhang, Chenglei Wu, Xin Yao, and Lifeng Sun. 2019. Comyco: Quality-aware adaptive video streaming via imitation learning. In Proceedings of the 27th ACM International Conference on Multimedia. ACM, 429--437.Google ScholarDigital Library
- Junchen Jiang et al. 2016. CFA: A practical prediction system for video QoE optimization. In Proceedings of the USENIX Symposium on Networked Systems Design and Implementation (NSDI’16). 137--150.Google Scholar
- Junchen Jiang et al. 2017. Pytheas: Enabling data-driven quality of experience optimization using group-based exploration-exploitation. In Proceedings of the USENIX Symposium on Networked Systems Design and Implementation (NSDI’17), Vol. 1. 3.Google Scholar
- Jessica Klein. 2018. Twitch Ended 2017 With 15 Million Daily Visitors, 27K Partnered Streamers. Retrieved February, 6, 2018 from https://www.tubefilter.com/2018/02/06/twitch-2017-year-in-review/.Google Scholar
- Hongqiang Harry Liu, Ye Wang, Yang Richard Yang, Hao Wang, and Chen Tian. 2012. Optimizing cost and performance for content multihoming. ACM Spec. Interest Group Data Commun. 42, 4 (2012), 371--382.Google ScholarDigital Library
- Xi Liu et al. 2012. A case for a coordinated internet video control plane. ACM SIGCOMM Comput. Commun. Rev. 42, 4 (2012), 359--370.Google ScholarDigital Library
- Scott M. Lundberg and Su-In Lee. 2017. A unified approach to interpreting model predictions. In Advances in Neural Information Processing Systems. 4765--4774.Google ScholarDigital Library
- Zoltán Ádám Mann. 2015. Allocation of virtual machines in cloud data centers—a survey of problem models and optimization algorithms. ACM Comput. Surv. 48, 1 (2015), 11.Google ScholarDigital Library
- Hongzi Mao, Ravi Netravali, and Mohammad Alizadeh. 2017. Neural adaptive video streaming with pensieve. In Proceedings of the Conference of the ACM Special Interest Group on Data Communication. ACM, 197--210.Google ScholarDigital Library
- Zili Meng, Jing Chen, Yaning Guo, Chen Sun, Hongxin Hu, and Mingwei Xu. 2019. PiTree: Practical implementation of ABR algorithms using decision trees. In Proceedings of the 27th ACM International Conference on Multimedia. ACM, 2431--2439.Google ScholarDigital Library
- Zili Meng, Minhu Wang, Mingwei Xu, Hongzi Mao, Jiasong Bai, and Hongxin Hu. 2019. Explaining deep learning-based networked systems. arXiv preprint arXiv:1910.03835 (2019).Google Scholar
- Volodymyr Mnih et al. 2015. Human-level control through deep reinforcement learning. Nature 518, 7540 (2015), 529.Google Scholar
- Haitian Pang, Cong Zhang, Fangxin Wang, Han Hu, Zhi Wang, Jiangchuan Liu, and Lifeng Sun. 2018. Optimizing personalized interaction experience in crowd-interactive livecast: A cloud-edge approach. In Proceedings of the 26th ACM International Conference on Multimedia. 1217--1225.Google ScholarDigital Library
- Marco Tulio Ribeiro, Sameer Singh, and Carlos Guestrin. 2016. Why should i trust you?: Explaining the predictions of any classifier. In Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. ACM, 1135--1144.Google ScholarDigital Library
- Stéphane Ross, Geoffrey Gordon, and Drew Bagnell. 2011. A reduction of imitation learning and structured prediction to no-regret online learning. In Proceedings of the 14th International Conference on Artificial Intelligence and Statistics. 627--635.Google Scholar
- David Silver, Guy Lever, Nicolas Heess, Thomas Degris, Daan Wierstra, and Martin Riedmiller. 2014. Deterministic policy gradient algorithms. In Proceedings of the 31st International Conference on International Conference on Machine Learning, Vol. 32. I--387.Google Scholar
- Mark Stemm and et al. 2000. A network measurement architecture for adaptive applications. In Proceedings of the IEEE International Conference on Computer Communications (INFOCOM’00). Vol. 1. IEEE, 285--294.Google Scholar
- Richard S. Sutton and Andrew G. Barto. 1998. Reinforcement Learning: An Introduction. Vol. 1. MIT Press, Cambridge, MA.Google ScholarDigital Library
- Fengxiao Tang, Bomin Mao, Zubair Md Fadlullah, Nei Kato, Osamu Akashi, Takeru Inoue, and Kimihiro Mizutani. 2017. On removing routing protocol from future wireless networks: A real-time deep learning approach for intelligent traffic control. IEEE Wireless Commun. 25, 1 (2017), 154--160.Google ScholarCross Ref
- Ruben Torres et al. 2011. Dissecting video server selection strategies in the youtube cdn. In Proceedings of the 2011 31st International Conference on Distributed Computing Systems (ICDCS’11). IEEE, 248--257.Google Scholar
- Andrew Trask et al. 2018. Neural arithmetic logic units. In Advances in Neural Information Processing Systems. 8046--8055.Google Scholar
- Jason Min Wang, Jun Zhang, and Brahim Bensaou. 2014. Content multi-homing: An alternative approach. In Proceedings of the 2014 IEEE International Conference on Communications (ICC’14). IEEE, 3118--3123.Google ScholarCross Ref
- Yifei Wei and et al. 2018. User scheduling and resource allocation in HetNets with hybrid energy supply: An actor-critic reinforcement learning approach. IEEE Trans. Wireless Commun. 17, 1 (2018), 680--692.Google ScholarDigital Library
- Cheng-Zhong Xu, Jia Rao, and Xiangping Bu. 2012. URL: A unified reinforcement learning approach for autonomic cloud management. J. Parallel Distrib. Comput. 72, 2 (2012), 95--105.Google ScholarDigital Library
- Zhiyuan Xu and et al. 2018. Experience-driven networking: A deep reinforcement learning based approach. arXiv preprint arXiv:1801.05757 (2018).Google Scholar
- Bo Yan and et al. 2017. LiveJack: Integrating CDNs and edge clouds for live content broadcasting. In Proceedings of the 2017 ACM on Multimedia Conference. ACM, 73--81.Google Scholar
- Francis Y. Yan, Jestin Ma, Greg D. Hill, Deepti Raghavan, Riad S. Wahby, Philip Levis, and Keith Winstein. 2018. Pantheon: The training ground for Internet congestion-control research. In Proceedings of the 2018 USENIX Annual Technical Conference (USENIX ATC’18). 731--743.Google Scholar
- Zhu Yifei and et al. 2017. When cloud meets uncertain crowd: An auction approach for crowdsourced livecast transcoding. In Proceedings of the 25th ACM International Conference on Multimedia. ACM, 1372--1380.Google Scholar
- Rui-Xiao Zhang, Tianchi Huang, Ming Ma, Haitian Pang, Xin Yao, Chenglei Wu, and Lifeng Sun. 2019. Enhancing the crowdsourced live streaming: A deep reinforcement learning approach. In Proceedings of the 29th ACM Workshop on Network and Operating Systems Support for Digital Audio and Video. ACM, 55--60.Google ScholarDigital Library
- Rui-Xiao Zhang, Tianchi Huang, Ming Ma, Haitian Pang, Xin Yao, Chenglei Wu, and Lifeng Sun. 2019. Enhancing the crowdsourced live streaming: A deep reinforcement learning approach. In Proceedings of the 29th ACM Workshop on Network and Operating Systems Support for Digital Audio and Video. ACM, 55--60.Google ScholarDigital Library
- Ying Zheng, Ziyu Liu, Xinyu You, Yuedong Xu, and Junchen Jiang. 2018. Demystifying deep learning in networking. In Proceedings of the 2nd Asia-Pacific Workshop on Networking. ACM, 1--7.Google ScholarDigital Library
Index Terms
- A Practical Learning-based Approach for Viewer Scheduling in the Crowdsourced Live Streaming
Recommendations
Enhancing the crowdsourced live streaming: a deep reinforcement learning approach
NOSSDAV '19: Proceedings of the 29th ACM Workshop on Network and Operating Systems Support for Digital Audio and VideoWith the growing demand for crowdsourced live streaming (CLS), how to schedule the large-scale dynamic viewers effectively among different Content Delivery Network (CDN) providers has become one of the most significant challenges for CLS platforms. ...
On crowdsourced interactive live streaming: a Twitch.tv-based measurement study
NOSSDAV '15: Proceedings of the 25th ACM Workshop on Network and Operating Systems Support for Digital Audio and VideoEmpowered by today's rich tools for media generation and collaborative production, the multimedia service paradigm is shifting from the conventional single source, to multi-source, to many sources, and now toward crowdsource. Such crowdsourced live ...
First Mile in Crowdsourced Live Streaming: A Content Harvest Network Approach
Thematic Workshops '17: Proceedings of the on Thematic Workshops of ACM Multimedia 2017Recent years have witnessed a rapid increase of crowdsourced live streaming (CLS): applications like Twitch.tv have attracted millions of daily active users. Content delivery in such crowdsourced live streaming involves two phases: 1) Video stream (i.e.,...
Comments