skip to main content
10.1145/3411502.3418427acmconferencesArticle/Chapter ViewAbstractPublication PagesccsConference Proceedingsconference-collections
research-article
Public Access

Remote Control of iOS Devices via Accessibility Features

Published: 09 November 2020 Publication History

Abstract

This paper presents an approach for remotely accessing and controlling mobile apps by leveraging a mobile platform's publicly exported accessibility features. This approach is implemented in a technique and accompanying tool called AirMochi. While AirMochi is designed to be platform-independent, our current implementation has focused on iOS, the significantly more challenging of the two dominant mobile platforms, for which access to apps' source code is generally not possible. We show that AirMochi places no restrictions on apps it can "plug into" and control, is able to handle a variety of scenarios, and imposes a negligible performance overhead.

References

[1]
2014. Veency, Cydia. https://cydia.saurik.com/info/veency/
[2]
2017. ReplayKit: Apple Developer Documentation. https://developer.apple.com/ documentation/replaykit
[3]
2018. WebRTC. https://webrtc.org/
[4]
2019. WebDriverAgent on GitHub. https://github.com/facebookarchive/ WebDriverAgent
[5]
2019. Wowza - 2019 Video Streaming Latency Report. https://www.wowza.com/ blog/2019-video-streaming-latency-report
[6]
2020. Appium: Mobile App Automation Made Awesome. http://appium.io/
[7]
2020. Eggplant Software. https://www.eggplantsoftware.com/
[8]
2020. Getevent: Android Open Source Project. https://source.android.com/ devices/input/getevent
[9]
2020. LongPressIos. https://developer.apple.com/documentation/uikit/ uilongpressgesturerecognizer/1616423-minimumpressduration
[10]
2020. Raspberry Pi OS (previously called Raspbian). https://www.raspberrypi. org/downloads/raspbian/
[11]
2020. Twilio Video: Video SDKs for iOS, Android, JavaScript and web-based video. https://www.twilio.com/video
[12]
2020. W3C WebDriver. https://w3c.github.io/webdriver/
[13]
Steven Arzt et almbox. 2014. FlowDroid: Precise Context, Flow, Field, Object-Sensitive and Lifecycle-Aware Taint Analysis for Android Apps. In Proceedings of the 35th ACM SIGPLAN Conference on Programming Language Design and Implementation (Edinburgh, United Kingdom) (PLDI '14). Association for Computing Machinery, New York, NY, USA, 259--269. https://doi.org/10.1145/2594291.2594299
[14]
C. Bernal-Cárdenas et almbox. 2020. Translating Video Recordings of Mobile App Usages into Replayable Scenarios. arXiv preprint arXiv:2005.09057 (2020).
[15]
D. Geist, M. Nigmatullin, and R. Bierens. 2016. Jailbreak/Root Detection Evasion Study on iOS and Android. MSc System and Network Engineering (2016).
[16]
L. Gomez, I. Neamtiu, T. Azim, and T. Millstein. 2013. RERAN: Timing- and touch-sensitive record and replay for Android. In 2013 35th International Conference on Software Engineering (ICSE). 72--81.
[17]
Heejin Kim et almbox. 2009. Performance testing based on test-driven development for mobile applications. In Proceedings of the 3rd International Conference on Ubiquitous Information Management and Communication. 612--617.
[18]
Y. K. Lee, J. Y. Bang, G. Safi, A. Shahbazian, Y. Zhao, and N. Medvidovic. 2017. A SEALANT for Inter-App Security Holes in Android. In 2017 IEEE/ACM 39th International Conference on Software Engineering (ICSE). 312--323.
[19]
J. W. Mickens et almbox. 2010. Crom: Faster Web Browsing Using Speculative Execution. In 2010 USENIX Symposium on Networked Systems Design and Implementation (NSDI), Vol. 10. 9--9.
[20]
T. A. Nguyen and C. Csallner. 2015. Reverse Engineering Mobile Application User Interfaces with REMAUI (T). In 2015 30th IEEE/ACM International Conference on Automated Software Engineering (ASE). 248--259.
[21]
Z. Qin, Y. Tang, E. Novak, and Q. Li. 2016. MobiPlay: A Remote Execution Based Record-and-Replay Tool for Mobile Applications. In 2016 IEEE/ACM 38th International Conference on Software Engineering (ICSE). 571--582.
[22]
Wenyu Wang et almbox. 2018. An empirical study of android test generation tools in industrial cases. In 2018 33rd IEEE/ACM International Conference on Automated Software Engineering (ASE). IEEE, 738--748.

Cited By

View all
  • (2024)ODACE: An Appium-based Testing Automation Platform for Android Mobile Devices Certification2024 IEEE International Conference on Software Testing, Verification and Validation Workshops (ICSTW)10.1109/ICSTW60967.2024.00060(301-308)Online publication date: 27-May-2024
  • (2021)UI Test Migration Across Mobile Platforms2021 36th IEEE/ACM International Conference on Automated Software Engineering (ASE)10.1109/ASE51524.2021.9678643(756-767)Online publication date: Nov-2021

Recommendations

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences
FEAST'20: Proceedings of the 2020 ACM Workshop on Forming an Ecosystem Around Software Transformation
November 2020
46 pages
ISBN:9781450380898
DOI:10.1145/3411502
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 09 November 2020

Permissions

Request permissions for this article.

Check for updates

Author Tags

  1. mobile application
  2. record-and-replay
  3. remote control

Qualifiers

  • Research-article

Funding Sources

Conference

CCS '20
Sponsor:

Acceptance Rates

Overall Acceptance Rate 4 of 4 submissions, 100%

Upcoming Conference

CCS '25

Contributors

Other Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

  • Downloads (Last 12 months)74
  • Downloads (Last 6 weeks)16
Reflects downloads up to 13 Feb 2025

Other Metrics

Citations

Cited By

View all
  • (2024)ODACE: An Appium-based Testing Automation Platform for Android Mobile Devices Certification2024 IEEE International Conference on Software Testing, Verification and Validation Workshops (ICSTW)10.1109/ICSTW60967.2024.00060(301-308)Online publication date: 27-May-2024
  • (2021)UI Test Migration Across Mobile Platforms2021 36th IEEE/ACM International Conference on Automated Software Engineering (ASE)10.1109/ASE51524.2021.9678643(756-767)Online publication date: Nov-2021

View Options

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

Login options

Figures

Tables

Media

Share

Share

Share this Publication link

Share on social media