Feng Qian
ABSTRACT
Flare is a practical system for streaming 360-degree videos on commodity mobile devices. It takes a viewport-adaptive approach, which fetches only portions of a panoramic scene that cover what a viewer is about to perceive. We conduct an IRB-approved user study where we collect head movement traces from 130 diverse users to gain insights on how to design the viewport prediction mechanism for Flare. We then develop novel online algorithms that determine which spatial portions to fetch and their corresponding qualities. We also innovate other components in the streaming pipeline such as decoding and server-side transmission. Through extensive evaluations (~400 hours' playback on WiFi and ~100 hours over LTE), we show that Flare significantly improves the QoE in real-world settings. Compared to non-viewport-adaptive approaches, Flare yields up to 18x quality level improvement on WiFi, and achieves high bandwidth reduction (up to 35%) and video quality enhancement (up to 4.9x) on LTE.
- 360 Google Spotlight Story: Help. https://www.youtube.com/watch?v=G-XZhKqQAHU.Google Scholar
- 360DEG Great Hammerhead Shark Encounter. https://www.youtube.com/watch?v=rG4jSz_2HDY.Google Scholar
- Android MediaCodec API. https://developer.android.com/reference/android/media/MediaCodec.html.Google Scholar
- Elephants on the Brink. https://www.youtube.com/watch?v=2bpICIClAIg.Google Scholar
- GT-R Drives First EVER 360 VR lap. https://www.youtube.com/watch?v=LD4XfM2TZ2k.Google Scholar
- Linux Packet Filtering framework. https://www.netfilter.org/documentation/HOWTO/packet-filtering-HOWTO.html.Google Scholar
- Mega Coaster: Get Ready for the Drop. https://www.youtube.com/watch?v=-xNN-bJQ4vI.Google Scholar
- Per-Title Encode Optimization. https://medium.com/netflix-techblog/per-title-encode-optimization-7e99442b62a2 .Google Scholar
- Pony Stable Playhouse for the Currys. https://www.youtube.com/watch?v=MWg1kjMmr3k.Google Scholar
- Under the hood: Building 360 video. https://code.facebook.com/posts/1638767863078802.Google Scholar
- Visit Hamilton Island in 360DEG Virtual Reality with Qantas. https://www.youtube.com/watch?v=lJype_TafRk.Google Scholar
- What Is Per-Title Encoding?. https://bitmovin.com/per-title-encoding/.Google Scholar
- YouTube live in 360 degrees encoder settings. https://support.google.com/youtube/answer/6396222.Google Scholar
- 360DEG, Angel Falls, Venezuela. https://www.youtube.com/watch?v=L_tqK4eqelA.Google Scholar
- 4G/LTE Bandwidth Logs. http://users.ugent.be/~jvdrhoof/dataset-4g/.Google Scholar
- Feel wimbledon with andy murray. https://www.youtube.com/watch?v=Krl6U15OERo.Google Scholar
- Qualcomm Snapdragon 835. https://www.qualcomm.com/products/snapdragon/processors/835.Google Scholar
- Skydive in 360DEG virtual reality via gopro. https://www.youtube.com/watch?v=S5XXsRuMPIU.Google Scholar
- Tomorrowland 2014 | 360 degrees of madness. https://www.youtube.com/watch?v=j81DDY4nvos.Google Scholar
- O. Abari, D. Bharadia, A. Duffield, and D. Katabi. Enabling High-Quality Untethered Virtual Reality. In Proceedings of NSDI 2017, pages 531--544. USENIX Association, 2017. Google ScholarDigital Library
- S. Afzal, J. Chen, and K. Ramakrishnan. Characterization of 360-Degree Videos. In Proceedings of the Workshop on Virtual Reality and Augmented Reality Network, pages 1--6. ACM, 2017. Google ScholarDigital Library
- A. Balachandran, V. Sekar, A. Akella, S. Seshan, I. Stoica, and H. Zhang. Developing a Predictive Model of Quality of Experience for Internet Video. In Proceedings of SIGCOMM 2013, pages 339--350. ACM, 2013. Google ScholarDigital Library
- Y. Bao, H. Wu, T. Zhang, A. A. Ramli, and X. Liu. Shooting a Moving Target: Motion-Prediction-Based Transmission for 360-Degree Videos. In Proceedings of Big Data 2016, pages 1161--1170. IEEE, 2016.Google ScholarCross Ref
- K. Boos, D. Chu, and E. Cuervo. FlashBack: Immersive Virtual Reality on Mobile Devices via Rendering Memoization. In Proceedings of MobiSys 2016, pages 291--304. ACM, 2016. Google ScholarDigital Library
- X. Corbillon, F. De~Simone, and G. Simon. 360-Degree Video Head Movement Dataset. In Proceedings of MMSys 2017. ACM, 2017. Google ScholarDigital Library
- X. Corbillon, G. Simon, A. Devlic, and J. Chakareski. Viewport-Adaptive Navigable 360-Degree Video Delivery. In Proceedings of ICC 2017. IEEE, 2017.Google Scholar
- C.-L. Fan, J. Lee, W.-C. Lo, C.-Y. Huang, K.-T. Chen, and C.-H. Hsu. Fixation Prediction for 360 Video Streaming in Head-Mounted Virtual Reality. In Proceedings of the Workshop on Network and Operating Systems Support for Digital Audio and Video, pages 67--72. ACM, 2017. Google ScholarDigital Library
- V. R. Gaddam, M. Riegler, R. Eg, C. Griwodz, and P. Halvorsen. Tiling in Interactive Panoramic Video: Approaches and Evaluation. IEEE Transactions on Multimedia, 18(9):1819--1831, 2016. Google ScholarDigital Library
- M. Graf, C. Timmerer, and C. Mueller. Towards bandwidth efficient adaptive streaming of omnidirectional video over HTTP: Design, implementation, and evaluation. In Proceedings of MMSys 2017, pages 261--271. ACM, 2017. Google ScholarDigital Library
- Y. Guo, F. Qian, Q. A. Chen, Z. M. Mao, and S. Sen. Understanding on-device bufferbloat for cellular upload. In Proceedings of IMC 2016 , pages 303--317. ACM, 2016. Google ScholarDigital Library
- B. Han, F. Qian, L. Ji, and V. Gopalakrishnan. MP-DASH: Adaptive Video Streaming Over Preference-Aware Multipath. In Proceedings of CoNEXT 2016, pages 129--143. ACM, 2016. Google ScholarDigital Library
- J. He, M. A. Qureshi, L. Qiu, J. Li, F. Li, and L. Han. Rubiks: Practical 360-Degree Streaming for Smartphones. In Proceedings of MobiSys 2018. ACM, 2018. Google ScholarDigital Library
- M. Hosseini and V. Swaminathan. Adaptive 360 VR video streaming: Divide and conquer. In Multimedia (ISM), 2016 IEEE International Symposium on, pages 107--110. IEEE, 2016.Google ScholarCross Ref
- T.-Y. Huang, R. Johari, N. McKeown, M. Trunnell, and M. Watson. A Buffer-Based Approach to Rate Adaptation: Evidence from a Large Video Streaming Service. In Proceedings of SIGCOMM 2014, pages 187--198. ACM, 2014. Google ScholarDigital Library
- H. Jiang, Y. Wang, K. Lee, and I. Rhee. Tackling bufferbloat in 3G/4G networks. In Proceedings of IMC 2012, pages 329--342. ACM, 2012. Google ScholarDigital Library
- J. Jiang, V. Sekar, and H. Zhang. Improving Fairness, Efficiency, and Stability in HTTP-Based Adaptive Video Streaming With Festive. In Proceedings of CoNEXT 2012, pages 97--108. ACM, 2012. Google ScholarDigital Library
- N. Jiang, V. Swaminathan, and S. Wei. Power Evaluation of 360 VR Video Streaming on Head Mounted Display Devices. In Proceedings of the 27th Workshop on Network and Operating Systems Support for Digital Audio and Video, pages 55--60. ACM, 2017. Google ScholarDigital Library
- C. Kelton, J. Ryoo, A. Balasubramanian, and S. R. Das. Improving User Perceived Page Load Times Using Gaze. In Proceedings of NSDI 2017, pages 545--559. USENIX Association, 2017. Google ScholarDigital Library
- Z. Lai, Y. C. Hu, Y. Cui, L. Sun, and N. Dai. Furion: Engineering high-quality immersive virtual reality on today's mobile devices. In Proceedings of MobiCom 2017, pages 409--421. ACM, 2017. Google ScholarDigital Library
- J. Li, Z. Wen, S. Li, Y. Zhao, B. Guo, and J. Wen. Novel tile segmentation scheme for omnidirectional video. In Proceedings of ICIP 2016, pages 370--374. IEEE, 2016.Google ScholarCross Ref
- T. Li, Q. Liu, and X. Zhou. Ultra-Low Power Gaze Tracking for Virtual Reality. In Proceedings of SenSys 2017. ACM, 2017.Google Scholar
- X. Liu, Q. Xiao, V. Gopalakrishnan, B. Han, F. Qian, and M. Varvello. 360 Innovations for Panoramic Video Streaming. In Proceedings of HotNets 2017. ACM, 2017. Google ScholarDigital Library
- W.-C. Lo, C.-L. Fan, J. Lee, C.-Y. Huang, K.-T. Chen, and C.-H. Hsu. 360 Video Viewing Dataset in Head-Mounted Virtual Reality. In Proceedings of MMSys 2017, pages 211--216. ACM, 2017. Google ScholarDigital Library
- H. Mao, R. Netravali, and M. Alizadeh. Neural Adaptive Video Streaming with Pensieve. In Proceedings of SIGCOMM 2017, pages 197--210. ACM, 2017. Google ScholarDigital Library
- X. Mi, F. Qian, and X. Wang. SMig: Stream Migration Extension For HTTP/2. In Proceedings of the 12th International on Conference on emerging Networking EXperiments and Technologies, pages 121--128. ACM, 2016. Google ScholarDigital Library
- K. Misra, A. Segall, M. Horowitz, S. Xu, A. Fuldseth, and M. Zhou. An overview of tiles in HEVC. IEEE Journal of selected topics in signal processing, 7(6):969--977, 2013.Google ScholarCross Ref
- S. Petrangeli, V. Swaminathan, M. Hosseini, and F. De~Turck. An HTTP/2-based adaptive streaming framework for 360DEG virtual reality videos. In Proceedings of MM 2017, pages 1--9. ACM, 2017. Google ScholarDigital Library
- F. Qian, B. Han, L. Ji, and V. Gopalakrishnan. Optimizing 360 video delivery over cellular networks. In Proceedings of the Workshop on All Things Cellular: Operations, Applications and Challenges, pages 1--6. ACM, 2016. Google ScholarDigital Library
- R. Real and J. M. Vargas. The probabilistic basis of Jaccard's index of similarity. Systematic biology, 45(3):380--385, 1996.Google ScholarCross Ref
- C. Saunders, A. Gammerman, and V. Vovk. Ridge Regression Learning Algorithm in Dual Variables. In Proceedings of ICML 1998, pages 515--521, 1998. Google ScholarDigital Library
- A. J. Smola and B. Schölkopf. A tutorial on support vector regression. Statistics and computing, 14(3):199--222, 2004. Google ScholarDigital Library
- K. K. Sreedhar, A. Aminlou, M. M. Hannuksela, and M. Gabbouj. Viewport-Adaptive Encoding and Streaming of 360-Degree Video for Virtual Reality Applications. In Proceedings of ISM 2016, pages 583--586. IEEE, 2016.Google ScholarCross Ref
- Z. Tan, Y. Li, Q. Li, Z. Zhang, Z. Li, and S. Lu. Enabling Mobile VR in LTE Networks: How Close Are We? In Proceedings of SIGMETRICS 2018. ACM, 2018. Google ScholarDigital Library
- H. Wang, V.-T. Nguyen, W. T. Ooi, and M. C. Chan. Mixing tile resolutions in tiled video: A perceptual quality assessment. In Proceedings of the Workshop on Network and Operating System Support on Digital Audio and Video, page~25. ACM, 2014. Google ScholarDigital Library
- C. Wu, Z. Tan, Z. Wang, and S. Yang. A Dataset for Exploring User Behaviors in VR Spherical Video Streaming. In Proceedings of MMSys 2017, pages 193--198. ACM, 2017. Google ScholarDigital Library
- M. Xiao, C. Zhou, V. Swaminathan, Y. Liu, and S. Chen. BAS-360: Exploring Spatial and Temporal Adaptability in 360-degree Videos over HTTP/2. In INFOCOM 2018-IEEE Conference on Computer Communications, IEEE. IEEE, 2018.Google ScholarCross Ref
- L. Xie, Z. Xu, Y. Ban, X. Zhang, and Z. Guo. 360ProbDASH: Improving QoE of 360 Video Streaming Using Tile-based HTTP Adaptive Streaming. In Proceedings of MM 2017, pages 315--323. ACM, 2017. Google ScholarDigital Library
- X. Xie and X. Zhang. POI360: Panoramic Mobile Video Telephony over LTE Cellular Networks. In Proceedings of CoNEXT 2017, pages 336--349. ACM, 2017. Google ScholarDigital Library
- X. Yin, A. Jindal, V. Sekar, and B. Sinopoli. A Control-Theoretic Approach for Dynamic Adaptive Video Streaming over HTTP. In Proceedings of SIGCOMM 2015, pages 325--338. ACM, 2015. Google ScholarDigital Library
- M. Yu, H. Lakshman, and B. Girod. A framework to evaluate omnidirectional video coding schemes. In Proceedings of the Symposium on Mixed and Augmented Reality (ISMAR) 2015, pages 31--36. IEEE, 2015. Google ScholarDigital Library
- A. Zare, A. Aminlou, M. M. Hannuksela, and M. Gabbouj. HEVC-compliant tile-based streaming of panoramic video for virtual reality applications. In Proceedings of MM 2016, pages 601--605. ACM, 2016. Google ScholarDigital Library
- C. Zhou, Z. Li, and Y. Liu. A Measurement Study of Oculus 360 Degree Video Streaming. In Proceedings of MMSys 2017. ACM, 2017. Google ScholarDigital Library
Index Terms
- Flare: Practical Viewport-Adaptive 360-Degree Video Streaming for Mobile Devices
Recommendations
Tile-Based Viewport-Adaptive Panoramic Video Streaming on Smartphones
MobiCom '18: Proceedings of the 24th Annual International Conference on Mobile Computing and NetworkingFlare is a practical system for streaming 360-degree videos on smartphones. It takes a viewport-adaptive approach, which fetches only portions of a panoramic scene that cover what a viewer is about to perceive. Flare consists of a novel framework for ...
Leveraging Sharing in Second Level Translation-Lookaside Buffers for Chip Multiprocessors
Traversing page table during virtual to physical address translation causes significant pipeline stalls when misses occur in the translation-lookaside buffer (TLB). To mitigate this penalty, we propose a fast, scalable, multi-level TLB organization that ...
A client-based adaptation framework for 360-degree video streaming
Highlights- The use of estimated bitrate/quality improves the viewport quality significantly.
Abstract360-degree video is one of the key components of Virtual Reality (VR) applications. 360-degree videos viewed on Head Mounted Displays can offer impressive viewing experiences to users. Yet, streaming of 360-degree videos over the ...
Comments