Sunjae Lee
ABSTRACT
Mobile apps offer a variety of features that greatly enhance user experience. However, users still often find it difficult to use mobile apps in the way they want. For example, it is not easy to use multiple apps simultaneously on a small screen of a smartphone. In this paper, we present A-Mash, a mobile platform that aims to simplify the way of interacting with multiple apps concurrently to the level of using a single app only. A key feature of A-Mash is that users can mash up the UIs of different existing mobile apps on a single screen according to their preferences. To this end, A-Mash 1) extracts UIs from unmodified existing apps (dynamic UI extraction) and 2) embeds extracted UIs from different apps into a single wrapper app (cross-process UI embedding), while 3) making all these processes hidden from the users (transparent execution environment). To the best of our knowledge, A-Mash is the first work to enable UIs of different unmodified legacy apps to seamlessly integrate and synchronize on a single screen, providing an illusion as if they were developed as a single app. A-Mash offers great potential for a number of useful usage scenarios. For instance, a user can mashup UIs of different IoT administration apps to create an all-in-one IoT device controller or one can mashup today's headlines from different news and magazine apps to craft one's own news headline collection. In addition, A-Mash can be extended to an AR space, in which users can map UI elements of different mobile apps to physical objects inside their AR scenes. Our evaluation of the A-Mash prototype implemented in Android OS demonstrates that A-Mash successfully supports the mashup of various existing mobile apps with little or no performance bottleneck. We also conducted in-depth user studies to assess the effectiveness of the A-Mash in real-world use cases.
- Tarfah Alrashed, Jumana Almahmoud, Amy X Zhang, and David R Karger. 2020. ScrAPIr: Making Web Data APIs Accessible to End Users. In Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems (CHI).Google Scholar
Digital Library
- Apple. 2022. Multitasking and Multiple Windows. https://developer.apple.com/design/human-interface-guidelines/ios/system-capabilities/multitasking/.Google Scholar
- Apple. 2022. Shortcuts User Guide. https://support.apple.com/guide/shortcuts/welcome/ios.Google Scholar
- Tanzirul Azim, Oriana Riva, and Suman Nath. 2016. ULink: Enabling User-Defined Deep Linking to App Content. In Proceedings of the 14th Annual International Conference on Mobile Systems, Applications, and Services (MobiSys).Google ScholarDigital Library
- John Brooke. 1995. SUS: A quick and dirty usability scale. Usability Eval. Ind. 189 (11 1995).Google Scholar
- Morgan Dixon and James Fogarty. 2010. Prefab: Implementing Advanced Behaviors Using Pixel-Based Reverse Engineering of Interface Structure. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI).Google ScholarDigital Library
- Rob Ennals, Eric Brewer, Minos Garofalakis, Michael Shadle, and Prashant Gandhi. 2007. Intel Mash Maker: Join the Web. ACM SIGMOD Record (2007).Google ScholarDigital Library
- Jun Fujima, Aran Lunzer, Kasper Hornbæk, and Yuzuru Tanaka. 2004. C3W: Clipping, Connecting and Cloning for the Web. In Proceedings of the 13th International World Wide Web Conference on Alternate Track Papers & Posters (WWW Alt.).Google ScholarDigital Library
- Google. 2022. Activity. https://developer.android.com/reference/android/app/Activity.Google Scholar
- Google. 2022. ARCore: Build new augmented reality experiences that seamlessly blend the digital and physical worlds. https://developers.google.com/ar.Google Scholar
- Google. 2022. Create a simple widget. https://developer.android.com/guide/topics/appwidgets.Google Scholar
- Google. 2022. Display. https://developer.android.com/reference/android/view/Display.Google Scholar
- Google. 2022. Flutter: Build apps for any screen. https://flutter.dev/.Google Scholar
- Google. 2022. Multi-Resume. https://source.android.com/devices/tech/display/multi_display/multi-resume.Google Scholar
- Google. 2022. Multi-window support. https://developer.android.com/guide/topics/large-screens/multi-window-support.Google Scholar
- Google. 2022. Picture-in-picture (PiP) support. https://developer.android.com/guide/topics/ui/picture-in-picture.Google Scholar
- Google. 2022. Profile battery usage with Batterystats and Battery Historian. https://developer.android.com/topic/performance/power/setup-battery-historian.Google Scholar
- Google. 2022. View. https://developer.android.com/reference/android/view/View.Google Scholar
- Donghwi Kim, Sooyoung Park, Jihoon Ko, Steven Y Ko, and Sung-Ju Lee. 2019. X-Droid: A Quick and Easy Android Prototyping Framework with a Single-App Illusion. In Proceedings of the 32nd Annual ACM Symposium on User Interface Software and Technology (UIST).Google ScholarDigital Library
- Sunjae Lee, Hayeon Lee, Hoyoung Kim, Sangmin Lee, Jeong Woon Choi, Yuseung Lee, Seono Lee, Ahyeon Kim, Jean Young Song, Sangeun Oh, et al. 2021. FLUID-XP: Flexible User Interface Distribution for Cross-Platform Experience. In Proceedings of the 27th Annual International Conference on Mobile Computing and Networking (MobiCom).Google ScholarDigital Library
- Tong Li, Mingyang Zhang, Hancheng Cao, Yong Li, Sasu Tarkoma, and Pan Hui. 2020. "What Apps Did You Use?": Understanding the Long-Term Evolution of Mobile App Usage. In Proceedings of The Web Conference 2020 (Taipei, Taiwan) (WWW '20). Association for Computing Machinery, New York, NY, USA, 66--76. Google ScholarDigital Library
- Toby Jia-Jun Li, Amos Azaria, and Brad A Myers. 2017. SUGILITE: Creating Multimodal Smartphone Automation by Demonstration. In Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems (CHI).Google Scholar
- Lin, James and Wong, Jeffrey and Nichols, Jeffrey and Cypher, Allen and Lau, Tessa A. 2009. End-User Programming of Mashups with Vegemite. In Proceedings of the 14th International Conference on Intelligent User Interfaces (IUI).Google Scholar
- Microsoft. 2022. HoloLens: Mixed Reality Technology for Business. https://www.microsoft.com/hololens.Google Scholar
- Sangeun Oh, Ahyeon Kim, Sunjae Lee, Kilho Lee, Dae R Jeong, Steven Y Ko, and Insik Shin. 2019. FLUID: Flexible User Interface Distribution for Ubiquitous Multi-Device Interaction. In The 25th Annual International Conference on Mobile Computing and Networking (MobiCom).Google ScholarDigital Library
- Meta Platforms. 2022. React Native: Learn once, write anywhere. https://reactnative.dev/.Google Scholar
- Onur Sahin, Assel Aliyeva, Hariharan Mathavan, Ayse Coskun, and Manuel Egele. 2019. RANDR: Record and Replay for Android Applications via Targeted Runtime Instrumentation. In Proceedings of the 34th IEEE/ACM International Conference on Automated Software Engineering (ASE).Google ScholarDigital Library
- Samsung. 2022. Set up and use Bixby Routines on your Galaxy phone. https://www.samsung.com/us/support/answer/ANS00083201/.Google Scholar
- Samsung. 2022. Using the Smart Pop-Up View my Galaxy Phone. https://www.samsung.com/au/support/mobile-devices/using-the-smart-pop-up-view/.Google Scholar
- Skia. 2022. Welcome to Skia: The 2D Graphics Library. https://skia.org/.Google Scholar
- Wolfgang Stuerzlinger, Olivier Chapuis, Dusty Phillips, and Nicolas Roussel. 2006. User Interface Façades: Towards Fully Adaptable User Interfaces. In Proceedings of the 19th Annual ACM Symposium on User Interface Software and Technology (UIST).Google ScholarDigital Library
- Desney S Tan, Brian Meyers, and Mary Czerwinski. 2004. WinCuts: Manipulating Arbitrary Window Regions for More Effective Use of Screen Space. In Proceedings of the Annual ACM Conference Extended Abstracts on Human Factors in Computing Systems (CHI EA).Google ScholarDigital Library
- Unity. 2022. Real-Time Development Platform. https://unity.com/.Google Scholar
- Jeffrey Wong and Jason I. Hong. 2007. Making Mashups with Marmite: Towards End-User Programming for the Web. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI).Google Scholar
- Jian Xu, Qingqing Cao, Aditya Prakash, Aruna Balasubramanian, and Donald E Porter. 2017. UIWear: Easily Adapting User Interfaces for Wearable Devices. In Proceedings of the 23rd Annual International Conference on Mobile Computing and Networking (MobiCom).Google ScholarDigital Library
- Jian Xu, Suwen Zhu, Aruna Balasubramanian, Xiaojun Bi, and Roy Shilkrot. 2018. Ultra-Low-Power Mode for Screenless Mobile Interaction. In Proceedings of the 31st Annual ACM Symposium on User Interface Software and Technology (UIST).Google ScholarDigital Library
- Qiang Xu, Jeffrey Erman, Alexandre Gerber, Zhuoqing Mao, Jeffrey Pang, and Shobha Venkataraman. 2011. Identifying Diverse Usage Behaviors of Smartphone Apps. In Proceedings of the 2011 ACM SIGCOMM Conference on Internet Measurement Conference (IMC).Google ScholarDigital Library
- Xiong Zhang and Philip J Guo. 2018. Fusion: Opportunistic Web Prototyping with UI Mashups. In Proceedings of the 31st Annual ACM Symposium on User Interface Software and Technology (UIST).Google ScholarDigital Library
- Jiahuan Zheng, Xin Peng, Jiacheng Yang, Huaqian Cai, Gang Huang, Ying Zhang, and Wenyun Zhao. 2017. CollaDroid: Automatic Augmentation of Android Application with Lightweight Interactive Collaboration. In Proceedings of the 2017 ACM Conference on Computer Supported Cooperative Work and Social Computing (CSCW).Google ScholarDigital Library
- Jiahuan Zheng, Liwei Shen, Xin Peng, Hongchi Zeng, and Wenyun Zhao. 2020. MashReDroid: enabling end-user creation of Android mashups based on record and replay. Science China Information Sciences 63, 10 (2020).Google Scholar
Index Terms
- A-mash: providing single-app illusion for multi-app use through user-centric UI mashup
Recommendations
Enhancing security enforcement on unmodified Android
SAC '13: Proceedings of the 28th Annual ACM Symposium on Applied ComputingAndroid OS have several security vulnerabilities. Most of existing proposals require extensive modification of Android kernel or application framework. So, they are not feasible for end users which use stock Android OS. In this paper, we present a novel ...
Mash Droid: An Approach to Mobile-Oriented Dynamic Services Discovery and Composition by In-App Search
ICWS '15: Proceedings of the 2015 IEEE International Conference on Web ServicesThe popularity of smartphones and tablet computers in recent years makes mobile apps burst. Mobile apps have become the main consumers of the Internet-based services. Compared to traditional applications in the desktop computing era, mobile devices with ...
Android vs iOS Security: A Comparative Study
ITNG '15: Proceedings of the 2015 12th International Conference on Information Technology - New GenerationsThe massive adoption of mobile devices by individuals as well as by organizations has brought forth many security concerns. Their significant abilities have resulted in their permeating use while correspondingly increasing their attractiveness as ...
Comments