skip to main content
10.1145/2999572.2999596acmconferencesArticle/Chapter ViewAbstractPublication PagesconextConference Proceedingsconference-collections
research-article

Enabling Automatic Protocol Behavior Analysis for Android Applications

Published: 06 December 2016 Publication History

Abstract

Android application is an important class on today's Internet. While understanding app-specific behavior is important for network operation and management, it is often difficult because it requires an in-depth application-layer protocol analysis due to the common use of HTTP(S) and standard data representations (e.g., JSON). This paper presents Extractocol, the first system to offer an automatic and comprehensive analysis of application protocol behaviors. Extractocol only uses Android application binary as input and accurately reconstructs HTTP transactions (request-response pairs) and identifies their message format and relationships using binary analysis. Our evaluation and in-depth case studies on commercial and open-source apps demonstrate that Extractocol provides high coverage and accurately characterizes network-related application behaviors.

References

[1]
Android library statistics - appbrain. http://www.appbrain.com/stats/libraries. Accessed Sep. 2015.
[2]
Bitblaze: Binary analysis for computer security. http://bitblaze.cs.berkeley.edu/. Accessed Jun, 2016.
[3]
Connecting to the network. https://developer.android.com/training/basics/network-ops/connecting.html. Accessed Jun. 2016.
[4]
Dynamic Site Acceleration explained. https://aiscaler.com/dsa-explained#prefetching. Accessed Sep. 2015.
[5]
Dynamic site accelerator. https://www.akamai.com/kr/ko/multimedia/documents/product-brief/dynamic-site-accelerator-product-brief.pdf. Accessed Oct. 2016.
[6]
Eliminate Blind Spots in SSL Encrypted Traffic. https://www.venafi.com/assets/pdf/sb/SSL_Visibility_Solution_Brief.pdf.
[7]
Extending prefetching to json and other objects. https://community.akamai.com/community/web-performance/blog/2015/02/19/extending-prefetching-to-json. Accessed Sep. 2015.
[8]
Free and open source android app repository. https://f-droid.org.
[9]
Generic android deobfuscator. https://github.com/CalebFenton/simplify. Accessed Jun. 2016.
[10]
How antivirus software works: Virus detection techniques. http://searchsecurity.techtarget.com/tip/How-antivirus-software-works-Virus-detection-techniques. Accessed Oct. 2016.
[11]
How dynamic site acceleration works: What at&t and akamai offer. http://blog.streamingmedia.com/2010/10/how-dynamic-site-acceleration-works-what-akamai-and-cotendo-offer.html. Accessed Sep. 2015.
[12]
A pattern based dalvik deobfuscator which uses limited execution to improve semantic analysis. https://github.com/CalebFenton/dex-oracle. Accessed Jun. 2016.
[13]
SSL Decryption. https://www.gigamon.com/products/technology/ssl-decryption. Accessed Oct. 2016.
[14]
SSL Encrypted Traffic Visibility and Management. https://www.bluecoat.com/products/ssl-encrypted-traffic-visibility-and-management. Accessed Oct. 2016.
[15]
Top applications on google play. https://www.appannie.com/apps/google-play/top-chart/united-states/application/. Accessed Sep. 2015.
[16]
An accuweather cloudlet answers a hail of data requests. https://www.yumpu.com/en/document/view/23074646/an-accuweather-cloudlet-answers-a-hail-of-data-requests, 2011.
[17]
Android async http clients: Volley vs retrofit. http://instructure.github.io/blog/2013/12/09/volley-vs-retrofit/, 2013.
[18]
Number of available android applications. http://www.appbrain.com/stats/number-of-android-apps, 2014.
[19]
Proxydroid - android apps on google play. https://play.google.com/store/apps/details?id=org.proxydroid&hl=en, Dec. 2014.
[20]
radio reddit v0.7. https://f-droid.org/repository/browse/?fdfilter=radio
[21]
reddit&fdid=com.radioreddit.android, 2014. An open-source radio reddit app.
[22]
Report: Apple iphone drives half of all mobile internet traffic. http://marketingland.com/report-apple-iphone-drives-half-mobile-internet-traffic-111129, 2014.
[23]
Reverse-engineering the kayak app with mitmproxy. http://www.shubhro.com/2014/12/18/reverse-engineering-kayak-mitmproxy/, Dec. 2014.
[24]
mitmproxy. https://mitmproxy.org/, 2015.
[25]
Proguard, android developers. http://developer.android.com/tools/help/proguard.html, 2015.
[26]
M. Amps. Reverse engineering shopify private apis. http://ma.rtin.so/reverse-engineering-shopify-private-apis, 2013.
[27]
J. Antunes and N. Neves. Automatically complementing protocol specifications from network traces. In European Workshop on Dependable Computing, 2011.
[28]
S. Arzt, S. Rasthofer, C. Fritz, E. Bodden, A. Bartel, J. Klein, Y. Le Traon, D. Octeau, and P. McDaniel. Flowdroid: Precise context, flow, field, object-sensitive and lifecycle-aware taint analysis for android apps. In ACM SIGPLAN PLDI, 2014.
[29]
A. Bartel, J. Klein, Y. Le Traon, and M. Monperrus. Dexpler: Converting android dalvik bytecode to jimple for static analysis with soot. In ACM SIGPLAN International Workshop on State of the Art in Java Program analysis, 2012.
[30]
M. A. Beddoe. Network protocol analysis using bioinformatics algorithms. http://www.4tphi.net/ awalters/PI/PI.html, 2004.
[31]
J. Caballero, P. Poosankam, C. Kreibich, and D. Song. Dispatcher: Enabling active botnet infiltration using automatic protocol reverse-engineering. In ACM CCS, 2009.
[32]
J. Caballero and D. Song. Rosetta: Extracting protocol semantics using binary analysis with applications to protocol replay and nat rewriting, 2007. Technical Report, UC Berkeley.
[33]
J. Caballero, H. Yin, Z. Liang, and D. Song. Polyglot: Automatic extraction of protocol message format using dynamic binary analysis. In ACM CCS, 2007.
[34]
Y. Cao, Y. Fratantonio, A. Bianchi, M. Egele, C. Kruegel, G. Vigna, and Y. Chen. Edgeminer: Automatically detecting implicit control flow transitions through the android framework. In NDSS, 2015.
[35]
E. Chin, A. P. Felt, K. Greenwood, and D. Wagner. Analyzing inter-application communication in Android. In ACM MobiSys, 2011.
[36]
B. Choi, J. Kim, and D. Han. Application-specific Acceleration Framework for Mobile Applications. In ACM SIGCOMM (poster session), 2016.
[37]
H. Choi, J. Kim, H. Hong, Y. Kim, J. Lee, and D. Han. Extractocol: Automatic extraction of application-level protocol behaviors for android applications. In ACM SIGCOMM (poster), pages 593--594. ACM, 2015.
[38]
P. M. Comparetti, G. Wondracek, C. Kruegel, and E. Kirda. Prospex: Protocol specification extraction. In IEEE Symposium Security and Privacy, 2009.
[39]
W. Cui, J. Kannan, and H. J. Wang. Discoverer: Automatic protocol reverse engineering from network traces. In USENIX Security, 2007.
[40]
W. Cui, V. Paxson, N. C. Weaver, and R. H. Katz. Protocol-independent adaptive replay of application dialog. In NDSS, 2006.
[41]
W. Cui, M. Peinado, K. Chen, H. J. Wang, and L. Irun-Briz. Tupni: Automatic reverse engineering of input formats. In ACM CCS, 2008.
[42]
S. Dai, A. Tongaonkar, X. Wang, A. Nucci, and D. Song. Networkprofiler: Towards automatic fingerprinting of android apps. In IEEE INFOCOM, 2013.
[43]
N. Derkach. Reverse engineering the private api: Hacking your couch. http://www.toptal.com/back-end/reverse-engineering-the-private-api-hacking-your-couch, 2014.
[44]
T. Dorr. Unofficial documentation of the Tesla Model S JSON API. http://docs.timdorr.apiary.io/, 2013.
[45]
P. Edholm. Reverse-engineering the kayak app with mitmproxy. http://www.nojitter.com/post/240153039/hp-and-microsoft-demo-openflowlync-applicationsoptimized-network. Apr. 2013.
[46]
M. Egele, D. Brumley, Y. Fratantonio, and C. Kruegel. An empirical study of cryptographic misuse in android applications. In ACM CCS, 2013.
[47]
W. Enck, P. Gilbert, B.-G. Chun, L. P. Cox, J. Jung, P. McDaniel, and A. N. Sheth. Taintdroid: An information-flow tracking system for realtime privacy monitoring on smartphones. In USENIX OSDI, 2010.
[48]
W. Enck, D. Octeau, P. McDaniel, and S. Chaudhuri. A study of android application security. In USENIX Security, 2011.
[49]
H. Falaki, D. Lymberopoulos, R. Mahajan, S. Kandula, and D. Estrin. A First Look at Traffic on Smartphones. In ACM Internet Measurement Conference, 2010.
[50]
J. François, H. Abdelnur, R. State, and O. Festor. Automated behavioral fingerprinting. In International Symposium on Recent Advances in Intrusion Detection, 2009.
[51]
Y. Fratantonio, A. Bianchi, W. Robertson, M. Egele, C. Kruegel, E. Kirda, and G. Vigna. On the Security and Engineering Implications of Finer-Grained Access Controls for Android Developers and Users. In International Conference on Detection of Intrusions and Malware, and Vulnerability Assessment, 2015.
[52]
C. Gibler, J. Crussell, J. Erickson, and H. Chen. Androidleaks: Automatically detecting potential privacy leaks in android applications on a large scale. In International conference on Trust and Trustworthy Computing, 2012.
[53]
M. I. Gordon, D. Kim, J. Perkins, L. Gilham, N. Nguyen, and M. Rinard. Information-Flow Analysis of Android Applications in DroidSafe. In NDSS, 2015.
[54]
M. Grace, Y. Zhou, Z. Wang, and X. Jiang. Systematic Detection of Capability Leaks in Stock Android Smartphones. In NDSS, 2012.
[55]
S. Hao, B. Liu, S. Nath, W. G. Halfond, and R. Govindan. Puma: Programmable ui-automation for large-scale dynamic analysis of mobile apps. In ACM Mobisys, 2014.
[56]
J. Huang, X. Zhang, L. Tan, P. Wang, and B. Liang. AsDroid: detecting stealthy behaviors in Android applications by user interface and program behavior contradiction. In International Conference on Software Engineering, 2014.
[57]
J. Khalid, A. Gember-Jacobson, R. Michael, A. Abhashkumar, and A. Akella. Paving the Way for NFV: Simplifying Middlebox Modifications using StateAlyzr. In USENIX NSDI, 2016.
[58]
J. Kim, Y. Yoon, K. Yi, J. Shin, and S Center. ScanDal: Static analyzer for detecting privacy leaks in android applications. In Mobile Security Technologies, 2012.
[59]
T. Krueger, N. Krämer, and K. Rieck. Asap: Automatic semantics-aware analysis of network payloads. In International Workshop on Privacy and Security Issues in Data Mining and Machine Learning, 2011.
[60]
C. Leita, K. Mermoud, and M. Dacier. Scriptgen: An automated script generation tool for honeyd. In Annual Computer Security Applications Conference, 2005.
[61]
O. Lhoták and L. Hendren. Scaling Java Points-to Analysis Using SPARK. In International Conference on Compiler Construction, 2003.
[62]
Z. Lin, X. Jiang, D. Xu, and X. Zhang. Automatic protocol format reverse engineering through context-aware monitored execution. In NDSS, 2008.
[63]
B. Liu, B. Liu, H. Jin, and R. Govindan. Efficient Privilege De-Escalation for Ad Libraries in Mobile Apps. In ACM MobiSys, 2015.
[64]
L. Lu, Z. Li, Z. Wu, W. Lee, and G. Jiang. CHEX: statically vetting Android apps for component hijacking vulnerabilities. In ACM CCS, 2012.
[65]
G. Maier, F. Schneider, and A. Feldmann. A First Look at Mobile Hand-held Device Traffic. In International Conference on Passive and Active Measurement, 2010.
[66]
mttkay. https://mttkay.github.io/blog/2013/08/25/functional-reactive-programming-on-android-with-rxjava/, 2013.
[67]
J. Musser. Reddit's secret api. http://www.programmableweb.com/news/reddits-secret-api/2008/11/25, 2008.
[68]
D. naylor, K. Schomp, M. Varvello, I. Leontiadis, D. L. Jeremy Balckburn, P. R. R. Konstantina Papaginnaki, and P. Steenkiste. Multi-Context TLS (mcTLS): Enabling Secure In-Network Functionality in TLS. In ACM SIGCOMM, 2015.
[69]
J. Newsome, D. Brumley, J. Franklin, and D. Song. Replayer: Automatic protocol replay by binary analysis. In ACM CCS, 2006.
[70]
D. Octeau, P. McDaniel, S. Jha, A. Bartel, E. Bodden, J. Klein, and Y. Le Traon. Effective inter-component communication mapping in Android with Epicc: an essential step towards holistic security analysis. In USENIX Security, 2013.
[71]
L. Pedrosa, A. Fogel, N. Kothari, R. Govindan, R. Mahajan, and T. Millstein. Analyzing protocol implementations for interoperability. In USENIX NSDI, 2015.
[72]
Z. A. Qazi, J. Lee, T. Jin, G. Bellala, M. Arndt, and G. Noubir. Application-awareness in SDN. In ACM SIGCOMM, 2013.
[73]
C. Qian, X. Luo, Y. Shao, and A. T. S. Chan. On Tracking Information Flows Through JNI in Android Applications. In IEEE/IFIP International Conference on Dependable Systems and Networks, 2014.
[74]
T. Reps, S. Horwitz, and M. Sagiv. Precise interprocedural dataflow analysis via graph reachability. In ACM SIGPLAN-SIGACT symposium on Principles of programming languages, 1995.
[75]
S. Saffron. How to reverse engineer the discourse api. https://meta.discourse.org/t/how-to-reverse-engineer-the-discourse-api/20576, 2014.
[76]
S. Singh, C. Estan, G. Varghese, and S. Savage. Automated worm fingerprinting. In USENIX OSDI, 2004.
[77]
D. Sounthiraraj, J. Sahs, G. Greenwood, Z. Lin, and L. Khan. SMV-HUNTER: Large Scale, Automated Detection of SSL/TLS Man-in-the-Middle Vulnerabilities in Android Apps. In NDSS, 2014.
[78]
A. Tongaonkar, R. Keralapura, and A. Nucci. Challenges in network application identification. In LEET, 2012.
[79]
A. Trifilò, S. Burschka, and E. Biersack. Traffic to protocol reverse engineering. In IEEE Symposium on Computational Intelligence for Security and Defense Applications, 2009.
[80]
N. Viennot, E. Garcia, and J. Nieh. A Measurement Study of Google Play. In ACM SIGMETRICS, 2014.
[81]
Y. Wang, Z. Zhang, D. D. Yao, B. Qu, and L. Guo. Inferring protocol state machine from network traces: A probabilistic approach. In International Conference on Applied Cryptography and Network Security, 2011.
[82]
F. Wei, S. Roy, X. Ou, and Robby. Amandroid: A Precise and General Inter-component Data Flow Analysis Framework for Security Vetting of Android Apps. In ACM CCS, 2014.
[83]
G. Wondracek, P. M. Comparetti, C. Kruegel, and E. Kirda. Automatic network protocol analysis. In NDSS, 2008.
[84]
J. Wright. Reverse Engineering the We Heart It API. http://jordan-wright.github.io/blog/2014/10/12/reverse-engineering-the-we-heart-it-api/, 2014.
[85]
Q. Xu, T. Andrews, Y. Liao, S. Miskovic, Z. M. Mao, M. Baldi, and A. Nucci. Flowr: A self-learning system for classifying mobile application traffic. In ACM SIGMETRICS, 2014.
[86]
Q. Xu, Y. Liao, S. Miskovic, Z. M. Mao, M. Baldi, A. Nucci, and T. Andrews. Automatic generation of mobile app signatures from traffic observations. In IEEE INFOCOM, 2015.
[87]
Z. Yang and M. Yang. LeakMiner: Detect information leakage on Android with static taint analysis. In World Congress on Software Engineering, 2012.
[88]
M. Zhang and H. Yin. AppSealer: Automatic Generation of Vulnerability-Specific Patches for Preventing Component Hijacking Attacks in Android Applications. In NDSS, 2014.
[89]
T. Zinner, M. Jarschel, A. Blenk, F. Wamser, and W. Kellerer. Dynamic application-aware resource management using Software-Defined Networking: Implementation prospects and challenges. In IEEE/IFIP Network Operations and Management Symposium, 2014.

Cited By

View all
  • (2023)IoTFlow: Inferring IoT Device Behavior at Scale through Static Mobile Companion App AnalysisProceedings of the 2023 ACM SIGSAC Conference on Computer and Communications Security10.1145/3576915.3623211(681-695)Online publication date: 15-Nov-2023
  • (2021)Software engineering techniques for statically analyzing mobile apps: research trends, characteristics, and potential for industrial adoptionJournal of Internet Services and Applications10.1186/s13174-021-00134-x12:1Online publication date: 23-Jul-2021
  • (2021)Automated Orchestration of Security Chains Driven by Process Learning*Communication Networks and Service Management in the Era of Artificial Intelligence and Machine Learning10.1002/9781119675525.ch12(289-319)Online publication date: 3-Sep-2021
  • Show More Cited By

Recommendations

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences
CoNEXT '16: Proceedings of the 12th International on Conference on emerging Networking EXperiments and Technologies
December 2016
524 pages
ISBN:9781450342926
DOI:10.1145/2999572
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 06 December 2016

Permissions

Request permissions for this article.

Check for updates

Author Tags

  1. android mobile application
  2. protocol behavior analysis
  3. reverse engineering

Qualifiers

  • Research-article

Funding Sources

Conference

CoNEXT '16
Sponsor:

Acceptance Rates

CoNEXT '16 Paper Acceptance Rate 30 of 160 submissions, 19%;
Overall Acceptance Rate 198 of 789 submissions, 25%

Contributors

Other Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

  • Downloads (Last 12 months)25
  • Downloads (Last 6 weeks)4
Reflects downloads up to 07 Mar 2025

Other Metrics

Citations

Cited By

View all
  • (2023)IoTFlow: Inferring IoT Device Behavior at Scale through Static Mobile Companion App AnalysisProceedings of the 2023 ACM SIGSAC Conference on Computer and Communications Security10.1145/3576915.3623211(681-695)Online publication date: 15-Nov-2023
  • (2021)Software engineering techniques for statically analyzing mobile apps: research trends, characteristics, and potential for industrial adoptionJournal of Internet Services and Applications10.1186/s13174-021-00134-x12:1Online publication date: 23-Jul-2021
  • (2021)Automated Orchestration of Security Chains Driven by Process Learning*Communication Networks and Service Management in the Era of Artificial Intelligence and Machine Learning10.1002/9781119675525.ch12(289-319)Online publication date: 3-Sep-2021
  • (2020)Lumos: Improving Smart Home IoT Visibility and Interoperability Through Analyzing Mobile Apps2020 IEEE 28th International Conference on Network Protocols (ICNP)10.1109/ICNP49622.2020.9259352(1-13)Online publication date: 13-Oct-2020
  • (2019)Looking from the mirrorProceedings of the 28th USENIX Conference on Security Symposium10.5555/3361338.3361418(1151-1167)Online publication date: 14-Aug-2019
  • (2019)Research on Data Mining of Permission-Induced Risk for Android IoT DevicesApplied Sciences10.3390/app90202779:2(277)Online publication date: 14-Jan-2019
  • (2019)LeakDoctorProceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies10.1145/33144153:1(1-25)Online publication date: 29-Mar-2019
  • (2018)Large-Scale Analysis of Remote Code Injection Attacks in Android AppsSecurity and Communication Networks10.1155/2018/24892142018Online publication date: 17-Apr-2018
  • (2018)APPxProceedings of the 14th International Conference on emerging Networking EXperiments and Technologies10.1145/3281411.3281416(27-40)Online publication date: 4-Dec-2018
  • (2018)Effective and Explainable Detection of Android Malware Based on Machine Learning AlgorithmsProceedings of the 2018 International Conference on Computing and Artificial Intelligence10.1145/3194452.3194465(35-40)Online publication date: 12-Mar-2018
  • Show More Cited By

View Options

Login options

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

Figures

Tables

Media

Share

Share

Share this Publication link

Share on social media