Naser AlDuaij
Alexander Van't Hof
Jason Nieh
ABSTRACT
As smartphones and tablets proliferate, there is a growing demand for multi-mobile computing, the ability to combine multiple mobile systems into more capable ones. We present M2, a system for multi-mobile computing that enables existing unmodified mobile apps to share and combine multiple devices, including cameras, displays, speakers, microphones, sensors, GPS, and input. M2 introduces a new data-centric approach that leverages higher-level device abstractions and hardware acceleration to efficiently share device data, not API calls. To support heterogeneous devices, M2 introduces device transformation, a new technique to mix and match different types of devices. Example transformations include combining multiple displays into a single larger display for better viewing, or substituting accelerometer for touchscreen input to provide a Nintendo Wii-like experience with existing mobile gaming apps. We have implemented M2 and show that it (1) operates across heterogeneous systems, including multiple versions of Android and iOS, (2) can enable unmodified Android apps to use multiple mobile devices in new and powerful ways, including supporting users with disabilities and better audio conferencing, and (3) can run apps across mobile systems with modest overhead and qualitative performance indistinguishable from using local device hardware.
- Naser AlDuaij, Alexander Van't Hof, and Jason Nieh. 2015. M2: Multi-Mobile Computing. Technical Report CUCS-005--15. Department of Computer Science, Columbia University.Google Scholar
- Naser AlDuaij, Alexander Van't Hof, and Jason Nieh. 2016. Heterogeneous Multi-Mobile Computing. Technical Report CUCS-008--16. Department of Computer Science, Columbia University.Google Scholar
- Naser AlDuaij, Alexander Van't Hof, and Jason Nieh. 2019. M2: Heterogeneous Multi-Mobile Computing . https://youtu.be/BzQ_YBA7kUU .Google Scholar
- Ardalan Amiri Sani, Kevin Boos, Min Hong Yun, and Lin Zhong. 2014a. Rio: A System Solution for Sharing I/O Between Mobile Systems. In Proceedings of the 12th Annual International Conference on Mobile Systems, Applications, and Services (MobiSys 2014). Bretton Woods, NH, 259--272.Google Scholar
- Ardalan Amiri Sani, Kevin Boos, Min Hong Yun, and Lin Zhong. 2014 b. Rio: A System Solution for Sharing I/O Between Mobile Systems . https://www.ruf.rice.edu/ mobile/rio.html .Google Scholar
- Jeremy Andrus, Naser AlDuaij, and Jason Nieh. 2017. Binary Compatible Graphics Support in Android for Running iOS Apps. In Proceedings of the 2017 ACM/IFIP/USENIX International Middleware Conference (Middleware 2017). Las Vegas, NV, 55--67. Google ScholarDigital Library
- Jeremy Andrus, Alexander Van't Hof, Naser AlDuaij, Christoffer Dall, Nicolas Viennot, and Jason Nieh. 2014. Cider: Native Execution of iOS Apps on Android . In Proceedings of the 19th International Conference on Architectural Support for Programming Languages and Operating Systems (ASPLOS 2014). Salt Lake City, UT, 367--381. Google ScholarDigital Library
- Apple Inc. 3D Touch - iOS - Apple Developer . https://developer.apple.com/ios/3d-touch/. Accessed: 2017-04--20.Google Scholar
- Apple, Inc. App Extensions - Apple Developer . https://developer.apple.com/app-extensions/. Accessed: 2018-03--21.Google Scholar
- Apple Inc. Apple - AirPlay - Play Content from iOS Devices on Apple TV. https://www.apple.com/airplay/. Accessed: 2014--12-07.Google Scholar
- Apple Inc. TV - Apple . https://www.apple.com/tv/. Accessed: 2018-08-07.Google Scholar
- Apple Inc. App Store - Apple. https://www.apple.com/ios/app-store/. Accessed: 2019-03--23.Google Scholar
- Apple Inc. 2019 b. Use Continuity to Connect Your Mac, iPhone, iPad, iPod touch, and Apple Watch . https://support.apple.com/en-us/HT204681 .Google Scholar
- Vaggelis Atlidakis, Jeremy Andrus, Roxana Geambasu, Dimitris Mitropoulos, and Jason Nieh. 2016. POSIX Abstractions in Modern Operating Systems: The Old, the New, and the Missing. In Proceedings of the 8th European Conference on Computer Systems (EuroSys 2016). London, UK, 19:1--17.Google ScholarDigital Library
- Ricardo Baratto, Leonard Kim, and Jason Nieh. 2005. THINC: A Virtual Display Architecture for Thin-Client Computing. In Proceedings of the 20th ACM Symposium on Operating Systems Principles (SOSP 2005). Brighton, UK, 277--290. Google ScholarDigital Library
- BlackBerry. BlackBerry Blend - Desktop Software for BlackBerry . https://us.blackberry.com/software/desktop/blackberry-blend . Accessed: 2017-03--15.Google Scholar
- Blender Foundation. Big Buck Bunny . https://peach.blender.org/. Accessed: 2018-08-07.Google Scholar
- Cesanta Software. 2019. Mongoose Embedded Web Server Library . https://github.com/cesanta/mongoose .Google Scholar
- Citrix Systems, Inc. Remote Access | GoToMyPC . https://www.gotomypc.com/remote-access/. Accessed: 2015-02--12.Google Scholar
- CyanogenMod Open-Source Community. 2016. CyanogenMod . https://web.archive.org/web/20161224194030/https://www.cyanogenmod.org/.Google Scholar
- James C. Corbett and et al. 2012. Spanner: Google's Globally-distributed Database. In Proceedings of the 10th USENIX Conference on Operating Systems Design and Implementation (OSDI 2012). Hollywood, CA, 251--264. Google ScholarDigital Library
- W. Keith Edwards, Mark W. Newman, Jana Sedivy, Trevor Smith, and Shahram Izadi. 2002. Challenge: Recombinant Computing and the Speakeasy Approach. In Proceedings of the 8th Annual International Conference on Mobile Computing and Networking (MobiCom 2002). Atlanta, GA, 279--286. Google ScholarDigital Library
- Darrell Etherington. 2013. Mosaic Lets You Weave A Single Display From Multiple iPhones And iPads, Offers SDK For Developers . https://techcrunch.com/2013/04/02/mosaic-lets-you-weave-a-single-display-from-multiple-iphones-and-ipads-offers-sdk-for-developers/.Google Scholar
- Google Inc. Chromecast - Google . https://www.google.com/chromecast . Accessed: 2017-04--20.Google Scholar
- Google Inc. Android Open Source Project . https://source.android.com/. Accessed: 2019-03--23.Google Scholar
- Google Inc. Google Play. https://play.google.com . Accessed: 2019-03--22.Google Scholar
- Alexander Van't Hof, Hani Jamjoom, Jason Nieh, and Dan Williams. 2015. Flux: Multi-Surface Computing in Android. In Proceedings of the 7th European Conference on Computer Systems (EuroSys 2015). Bordeaux, France, 24:1--17. Google ScholarDigital Library
- Alexander Van't Hof and Jason Nieh. 2019. AnDrone: Virtual Drone Computing in the Cloud . In Proceedings of the 11th European Conference on Computer Systems (EuroSys 2019). Dresden, Germany, 6:1--16.Google Scholar
- Greg Humphreys, Mike Houston, Ren Ng, Randall Frank, Sean Ahern, Peter D. Kirchner, and James T. Klosowski. 2002. Chromium: A Stream-processing Framework for Interactive Rendering on Clusters. In Proceedings of the 29th Annual Conference on Computer Graphics and Interactive Techniques (SIGGRAPH 2002). San Antonio, TX, 693--702. Google ScholarDigital Library
- Internet Engineering Task Force (IETF). 2011. RFC 6143 - The Remote Framebuffer Protocol . https://tools.ietf.org/html/rfc6143 .Google Scholar
- Internet Engineering Task Force (IETF). 2013. RFC 6762 - Multicast DNS . https://tools.ietf.org/html/rfc6762 .Google Scholar
- Yu-Wen Jong, Pi-Cheng Hsiu, Sheng-Wei Cheng, and Tei-Wei Kuo. 2016. A Semantics-aware Design for Mounting Remote Sensors on Mobile Systems. In Proceedings of the 53rd Annual Design Automation Conference (DAC 2016). Austin, TX, 140:1--6. Google ScholarDigital Library
- Lorenzo Keller, Anh Le, Blerim Cici, Hulya Seferoglu, Christina Fragouli, and Athina Markopoulou. 2012. MicroCast: Cooperative Video Streaming on Smartphones. In Proceedings of the 10th International Conference on Mobile Systems, Applications, and Services (MobiSys 2012). Low Wood Bay, Lake District, UK, 57--70. Google ScholarDigital Library
- Joeng Kim, Ricardo Baratto, and Jason Nieh. 2006. pTHINC: A Thin-Client Architecture for Mobile Wireless Web. In Proceedings of the 15th International World Wide Web Conference (WWW 2006). Edinburgh, Scotland, 143--152. Google ScholarDigital Library
- Qingkai Kong, Qin Lv, and Richard M. Allen. 2019. Earthquake Early Warning and Beyond: Systems Challenges in Smartphone-based Seismic Network. In Proceedings of the 20th International Workshop on Mobile Computing Systems and Applications (HotMobile 2019). Santa Cruz, CA, 57--62.Google ScholarDigital Library
- Richard Lawler. 2018. iOS 12 Developer Beta Points to Bezel-Less iPad with Face ID . https://www.engadget.com/2018/08/02/ipad-pro-2-ios-12-beta-leak-bezel-faceid/.Google Scholar
- Kai Li and Paul Hudak. 1986. Memory Coherence in Shared Virtual Memory Systems. In Proceedings of the 5th Annual ACM Symposium on Principles of Distributed Computing (PODC 1986). Calgary, Alberta, Canada, 229--239. Google ScholarDigital Library
- Yong Li and Wei Gao. 2017. Interconnecting Heterogeneous Devices in the Personal Mobile Cloud. In Proceedings of the 36th IEEE Conference on Computer Communications (INFOCOM 2017). Atlanta, GA, 1--9.Google ScholarCross Ref
- The Linux Information Project. 2006. An Introduction to X by the Linux Information Project (LINFO) . http://www.linfo.org/x.html .Google Scholar
- Alexander Löffler, Luciano Pica, Hilko Hoffmann, and Philipp Slusallek. 2012. Networked Displays for VR Applications: Display as a Service (DaaS). In Virtual Environments 2012: Proceedings of Joint Virtual Reality Conference of ICAT, EuroVR and EGVE (JVRC) (ICAT/EGVE/EuroVR 2012). Madrid, Spain, 37--44.Google Scholar
- Wenguang Mao, Jian He, and Lili Qiu. 2016. CAT: High-precision Acoustic Motion Tracking. In Proceedings of the 22nd Annual International Conference on Mobile Computing and Networking (MobiCom 2016). New York, NY, 69--81. Google ScholarDigital Library
- James Alastair McLaughlin. 2018. Very Low Footprint JSON Parser Written in Portable ANSI C . https://github.com/udp/json-parser .Google Scholar
- Microsoft Corporation. Skype | Communication Tool for Free Calls and Chat . https://www.skype.com . Accessed: 2019-03--21.Google Scholar
- Microsoft Corporation. Windows Continuum for Windows 10 Phones and Mobile . https://www.microsoft.com/en-us/windows/continuum . Accessed: 2017-03--15.Google Scholar
- Microsoft Corporation. 2008. SenseWeb - Microsoft Research . https://www.microsoft.com/en-us/research/project/senseweb/.Google Scholar
- Microsoft Corporation. 2018. Remote Desktop Protocol (Windows) . https://msdn.microsoft.com/en-us/library/aa383015.aspx .Google Scholar
- Monsoon Solutions, Inc. Monsoon Solutions | Printed Circuit Board Design & Manufacturing . https://www.msoon.com . Accessed: 2019-03--21.Google Scholar
- Sungwon Nam, Sachin Deshpande, Venkatram Vishwanath, Byungil Jeong, Luc Renambot, and Jason Leigh. 2010. Multi-application Inter-tile Synchronization on Ultra-high-resolution Display Walls. In Proceedings of the 1st Annual ACM SIGMM Conference on Multimedia Systems (MMSys 2010). Phoenix, AZ, 145--156. Google ScholarDigital Library
- Jason Nieh and S. Jae Yang. 2000. Measuring the Multimedia Performance of Server-Based Computing. In Proceedings of the 10th International Workshop on Network and Operating System Support for Digital Audio and Video. Chapel Hill, NC, 55--64.Google Scholar
- Jason Nieh, S. Jae Yang, and Naomi Novik. 2003. Measuring Thin-Client Performance Using Slow-Motion Benchmarking . ACM Transactions on Computer Systems (TOCS), Vol. 21, 1 (Feb. 2003), 87--115. Google ScholarDigital Library
- Nintendo Co., Ltd. Nintendo Switch . https://www.nintendo.com/switch . Accessed: 2017-03--15.Google Scholar
- Sangeun Oh, Hyuck Yoo, Dae R. Jeong, Duc Hoang Bui, and Insik Shin. 2017. Mobile Plus: Multi-device Mobile Platform for Cross-device Functionality Sharing. In Proceedings of the 15th Annual International Conference on Mobile Systems, Applications, and Services (MobiSys 2017). Niagara Falls, NY, 332--344. Google ScholarDigital Library
- Takashi Ohta and Jun Tanaka. 2012. Pinch: An Interface That Relates Applications on Multiple Touch-screen by 'Pinching' Gesture. In Proceedings of the 9th International Conference on Advances in Computer Entertainment (ACE 2012). Kathmandu, Nepal, 320--335. Google ScholarDigital Library
- OpenCV team. OpenCV library . https://www.opencv.org . Accessed: 2017-04--18.Google Scholar
- OpenSignal. 2015. Android Fragmentation Visualized . https://www.opensignal.com/sites/opensignal-com/files/data/reports/global/data-2015-08/2015_08_fragmentation_report.pdf .Google Scholar
- PassMark Software, Inc. PassMark PerformanceTest - Android Apps on Google Play. https://play.google.com/store/apps/details?id=com.passmark.pt_mobile . Accessed: 2015-03--10.Google Scholar
- Chunyi Peng, Guobin Shen, Yongguang Zhang, Yanlin Li, and Kun Tan. 2007. BeepBeep: A High Accuracy Acoustic Ranging System Using COTS Mobile Devices. In Proceedings of the 5th International Conference on Embedded Networked Sensor Systems (SenSys 2007). Sydney, Australia, 1--14.Google ScholarDigital Library
- Federico Perazzi, Alexander Sorkine-Hornung, Henning Zimmer, Peter Kaufmann, Oliver Wang, Sharon Watson, and Markus H. Gross. 2015. Panoramic Video from Unstructured Camera Arrays. Computer Graphics Forum, Vol. 34, 2 (May 2015), 57--68. Google ScholarDigital Library
- Pocketnow. 2013. Samsung Group Play Video Sharing Demo at IFA 2013 . https://www.youtube.com/watch?v=hhvu9ugtVY4 .Google Scholar
- Pushbullet. Pushbullet - SMS on PC - Android Apps on Google Play . https://play.google.com/store/apps/details?id=com.pushbullet.android . Accessed: 2017-03--15.Google Scholar
- Claire Reilly. 2018. Samsung's New Galaxy Phone Patent Is a Bezel-Less, Notch-Free Vision of the Future . https://www.cnet.com/news/samsung-galaxy-phone-patent-is-a-bezel-less-notch-free-slice-of-the-future/.Google Scholar
- Kay Römer. 2001. Time Synchronization in Ad Hoc Networks. In Proceedings of the 2nd ACM International Symposium on Mobile Ad Hoc Networking and Computing (MobiHoc 2001). Long Beach, CA, 173--182. Google ScholarDigital Library
- Sand Studio. AirDroid: Remote Access and File - Android Apps on Google Play . https://play.google.com/store/apps/details?id=com.sand.airdroid . Accessed: 2017-03--15.Google Scholar
- Arne Schmitz, Ming Li, Volker Schönefeld, and Leif Kobbelt. 2010. Ad-Hoc Multi-Displays for Mobile Interactive Applications. In Proceedings of the 31st Annual Conference of the European Association for Computer Graphics (Eurographics 2010). Norrkö ping, Sweden, 45--52.Google Scholar
- Securax LTD. Zoiper Audio Latency Benchmark - Android Apps on Google Play. https://play.google.com/store/apps/details?id=com.zoiper.audiolatency.app . Accessed: 2015-03-05.Google Scholar
- Guobin Shen, Yanlin Li, and Yongguang Zhang. 2007. MobiUS: Enable Together-viewing Video Experience Across Two Mobile Devices. In Proceedings of the 5th International Conference on Mobile Systems, Applications and Services (MobiSys 2007). San Juan, Puerto Rico, 30--42.Google ScholarDigital Library
- Ben Shneiderman and Catherine Plaisant. 2004. Designing the User Interface: Strategies for Effective Human-Computer Interaction (4th Edition) .Pearson Addison Wesley, Boston, MA. Google ScholarDigital Library
- Liz Stinson. 2017. What's the Big Deal With All These Bezel-Free Phones? https://www.wired.com/story/whats-the-big-deal-with-all-these-bezel-free-phones/.Google Scholar
- Mélodie Vidal, Andreas Bulling, and Hans Gellersen. 2013. Pursuits: Spontaneous Interaction with Displays Based on Smooth Pursuit Eye Movement and Moving Targets. In Proceedings of the 2013 ACM International Joint Conference on Pervasive and Ubiquitous Computing (UbiComp 2013). Zurich, Switzerland, 439--448. Google ScholarDigital Library
- VideoLAN Organization. VideoLAN - Official page for VLC media player . https://www.videolan.org . Accessed: 2016-05--10.Google Scholar
- Wei Wang, Alex X. Liu, and Ke Sun. 2016. Device-free Gesture Tracking Using Acoustic Signals. In Proceedings of the 22nd Annual International Conference on Mobile Computing and Networking (MobiCom 2016). New York, NY, 82--94. Google ScholarDigital Library
- Roy Want, Trevor Pering, Shivani Sud, and Barbara Rosario. 2008. Dynamic Composable Computing. In Proceedings of the 9th Workshop on Mobile Computing Systems and Applications (HotMobile 2008). Napa Valley, CA, 17--21.Google ScholarDigital Library
- Wi-Fi Alliance. Wi-Fi Direct | Wi-Fi Alliance . https://www.wi-fi.org/discover-wi-fi/wi-fi-direct . Accessed: 2019-03--23.Google Scholar
- Raymond Wong. 2018. If You Hate the iPhone X 'Notch,' These Phones Have Some Good News for You . https://mashable.com/2018/02/26/smartphones-true-bezel-less-displays/.Google Scholar
- X.Org Foundation. X.Org . https://www.x.org . Accessed: 2015-02--27.Google Scholar
- S. Jae Yang, Jason Nieh, Shilpa Krishnappa, Aparna Mohla, and Mahdi Sajjadpour. 2003. Web Browsing Performance of Wireless Thin-Client Computing. In Proceedings of the 12th International World Wide Web Conference (WWW 2003). Budapest, Hungary, 68--79. Google ScholarDigital Library
- S. Jae Yang, Jason Nieh, and Naomi Novik. 2001. Measuring Thin-Client Performance Using Slow-Motion Benchmarking. In Proceedings of the 2001 USENIX Annual Technical Conference (USENIX ATC 2001). Boston, MA, 35--49. Google ScholarDigital Library
- S. Jae Yang, Jason Nieh, Matt Selsky, and Nikhil Tiwari. 2002. The Performance of Remote Display Mechanisms for Thin-Client Computing. In Proceedings of the 2002 USENIX Annual Technical Conference (USENIX ATC 2002). Monterey, CA, 131--146. Google ScholarDigital Library
- Katie Young. 2017. Digital Consumers Own 3.2 Connected Devices - GlobalWebIndex Blog. https://blog.globalwebindex.com/chart-of-the-day/digital-consumers-own-3-point-2-connected-devices/.Google Scholar
- Yanxia Zhang, Andreas Bulling, and Hans Gellersen. 2013. SideWays: A Gaze Interface for Spontaneous Interaction with Situated Displays. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI 2013). Paris, France, 851--860. Google ScholarDigital Library
Index Terms
- Heterogeneous Multi-Mobile Computing
Recommendations
Tap: an app framework for dynamically composable mobile systems
MobiSys '21: Proceedings of the 19th Annual International Conference on Mobile Systems, Applications, and ServicesAs smartphones and tablets have become ubiquitous, there is a growing demand for apps that can enable users to collaboratively use multiple mobile systems. We present Tap, a framework that makes it easy for users to dynamically compose collections of ...
Heterogeneous Multi-Mobile Computing (video)
MobiSys '19: Proceedings of the 17th Annual International Conference on Mobile Systems, Applications, and ServicesAs smartphones and tablets proliferate, there is a growing demand for multi-mobile computing [1, 2], the ability to combine multiple commodity mobile systems into more capable ones, including using multiple hardware devices such as cameras, displays, ...
Binary compatible graphics support in Android for running iOS apps
Middleware '17: Proceedings of the 18th ACM/IFIP/USENIX Middleware ConferenceMobile apps make extensive use of GPUs on smartphones and tablets to access Web content. To support pervasive Web content, we introduce three key OS techniques for binary graphics compatibility necessary to build a real-world system to run iOS and ...
Comments