Skip to main content

Advertisement

Springer Nature Link
Log in

Benchmark applications used in mobile cloud computing research: a systematic mapping study

  • Published:
The Journal of Supercomputing Aims and scope Submit manuscript

Abstract

Mobile cloud computing (MCC) integrates mobile computing and cloud computing aiming to extend the capabilities of mobile devices through offloading techniques. In MCC, many controlled experiments have been performed using mobile applications as benchmarks. Usually, these applications are used to validate proposed algorithms, architectures or frameworks. The task of choosing a specific benchmark to evaluate MCC proposals is difficult because there is no standard applications list. This paper presents a systematic mapping study for benchmarks used in MCC research. Taking 5 months of work, we have read 763 papers from MCC field. We catalogued the applications and characterized them considering three facets: category (e.g., games, imaging tools); evaluated resource (e.g., time, energy); and platform (e.g., Android, iPhone). The mapping study evidences research gaps and research trends. Providing a list of downloadable standardized benchmarks, this work can aid better choices to guide more reliable research studies since the same application could be used for different scientific purposes.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15

Similar content being viewed by others

Notes

  1. http://www.android.com/.

  2. http://www.windowsphone.com/.

  3. https://www.apple.com/iPhone/.

  4. http://www.android-x86.org/.

  5. http://maemo.org/.

  6. http://br.blackberry.com/.

References

  1. Armbrust M, Fox A, Griffith R, Joseph AD, Katz R, Konwinski A, Lee G, Patterson D, Rabkin A, Stoica I, Zaharia M (2010) A view of cloud computing. Commun ACM 53(4):50–58

    Article  Google Scholar 

  2. Brasilino Leite Neto C, De Carvalho Filho P, Nobrega Duarte A (2013) A systematic mapping study on fault management in cloud computing. In: Parallel and Distributed Computing, Applications and Technologies (PDCAT), 2013 International Conference on, pp 332–337

  3. Budgen D, Turner M, Brereton P, Kitchenham B (2008) Using mapping studies in Software Engineering. In: Proc. of PPIG 2008, pp 195–204. Lancaster University

  4. Cagalaban G, Kim S, Kim M (2012) A mobile device-based virtualization technique for m2m communication in cloud computing security. In: Kim T-h, Stoica A, Fang W-c, Vasilakos T, Villalba J, Arnett K, Khan M, Kang B-H, (eds) Computer Applications for Security, Control and System Engineering, vol. 339 of Communications in Computer and Information Science, pp 160–167. Springer, Berlin, Heidelberg

  5. Chang RS, Gao J, Gruhn V, He J, Roussos G, Tsai WT (2013) Mobile cloud computing research - issues, challenges and needs. In: Service Oriented System Engineering (SOSE), 2013 IEEE 7th International Symposium on, pp 442–453

  6. Chun B-G, Ihm S, Maniatis P, Naik M, Patti A (2013) Clonecloud: Elastic execution between mobile device and cloud. In: Proc. of the Sixth Conference on Computer Systems, EuroSys ’11, pp 301–314. New York (ACM 2011)

  7. Cidon A, London TM, Katti S, Kozyrakis C, Rosenblum M (2011) Mars: adaptive remote execution for multi-threaded mobile devices. In: Proc. of the 3rd ACM SOSP Workshop on Networking, Systems, and Applications on Mobile Handhelds, MobiHeld ’11, pp 1:1–1:6. New York (ACM 2011)

  8. Colombo-Mendoza LO, Alor AG, Valencia-garcía R (2014) MobiCloUP!: a PaaS for cloud services-based mobile applications. Automated Software Engineering 21(3):391–437

  9. Cuervo E, Balasubramanian A, Cho D-k, Wolman A, Saroiu S, Chandra R, Bahl P (2010) Maui: making smartphones last longer with code offload. In: Proc. of the 8th International Conference on Mobile Systems, Applications, and Services, MobiSys ’10, pp 49–62. New York (ACM 2010)

  10. da Silva CMR, da Silva JLC, Rodrigues RB, Nascimento LM, Garcia VC (2013) Systematic mapping study on security threats in cloud computing. CoRR. arXiv:1303.6782

  11. Dinh HT, Lee C, Niyato D, Wang P (2013) A survey of mobile cloud computing: architecture, applications, and approaches. Wirel Commun Mobile Comput 13(18):1587–1611

    Article  Google Scholar 

  12. Eom H, Juste PS, Figueiredo R, Tickoo O, Illikkal R, Iyer R (2012) SNARF: a social networking-inspired accelerator remoting framework, pp 29–34

  13. Eom H, Juste PS, Figueiredo R, Tickoo O, Illikkal R, Iyer R (2013) Machine learning-based runtime scheduler for mobile offloading framework. In: 2013 IEEE/ACM 6th International Conference on Utility and Cloud Computing, pp 17–25

  14. Eom H, Juste PS, Figueiredo R, Tickoo O, Illikkal R, Iyer R (2013) OpenCL-based remote offloading framework for trusted mobile cloud computing. In: 2013 International Conference on Parallel and Distributed Systems, pp 240–248

  15. Ferber M, Rauber T (2012) Mobile cloud computing in 3G cellular networks using pipelined tasks. In: Proc. of the First European Conference on Service-Oriented and Cloud Computing, ESOCC’12, pp 192–199. Springer, Berlin, Heidelberg

  16. Fernando N, Loke S, Rahayu W (2013) Honeybee: a programming framework for mobile crowd computing. In: Zheng K, Li M, Jiang H (eds) mobile and ubiquitous systems: computing, networking, and services, vol. 120 of Lecture Notes of the Institute for Computer Sciences, Social Informatics and Telecommunications Engineering, pp 224–236. Springer, Berlin, Heidelberg

  17. Flores H, Srirama SN, Paniagua C (2012) Towards mobile cloud applications: offloading resource-intensive tasks to hybrid clouds. Int J Pervasive Comput 8(4):344–367

    Article  Google Scholar 

  18. Giurgiu I, Riva O, Alonso G (2012) From clouds to mobile devices, pp 394–414

  19. Gordon MS, Jamshidi DA, Mahlke S, Mao ZM, Chen X (2012) Comet: code offload by migrating execution transparently. In: Proceedings of the 10th USENIX Conference on Operating Systems Design and Implementation, OSDI’12, pp 93–106. USENIX Association, Berkeley

  20. Guan L, Ke X, Song M, Song J (2011) A survey of research on mobile cloud computing. In: Computer and Information Science (ICIS), 2011 IEEE/ACIS 10th Int. Conf on, pp 387–392

  21. Hassan M, Chen S (2012) Mobile mapreduce: minimizing response time of computing intensive mobile applications. In: Zhang J, Wilkiewicz J, Nahapetian A (eds) Mobile computing, applications, and services, vol. 95 of Lecture Notes of the Institute for Computer Sciences, Social Informatics and Telecommunications Engineering, pp 41–59. Springer, Berlin, Heidelberg

  22. Hassan MA, Chen S (2012) An investigation of different computing sources for mobile application outsourcing on the road. In: Mobile wireless middleware, operating systems, and applications, pp 153–166. Springer, New York

  23. Hill S (2014) Android 4.4 vs. ios 7 vs. windows phone 8: which smartphone os is best? http://www.digitaltrends.com/mobile/best-smartphone-os/. Accessed on 13 October 2014

  24. Huang C-C, Huang J-L, Tsai C-L, Wu G-Z, Chen C-M, Lee W-C (2013) Energy-efficient and cost-effective web API invocations with transfer size reduction for mobile mashup applications. Wirel Netw 20(3):361–378

    Article  Google Scholar 

  25. Jain P, Kabra R, Rustagi S, Bansal T, Patel D, Raychoudhury V (2013) Mc2: On-the-fly mobile compute cloud for computational intensive task. In: Proc. of the 5th IBM Collaborative Academia Research Exchange, I-CARE ’13, pp 7:1–7:4. ACM, New York

  26. Jhingut MZ, Ghoorun IM, Nagowah SD, Moloo R, Nagowah L (2010) Design and development of 3d mobile games. In: Proceedings of the 2010 Third International Conference on Advances in Computer-Human Interactions, ACHI ’10, pp 119–124. IEEE Computer Society, Washington, DC

  27. Kakadia D (2013) MECCA: mobile, efficient cloud computing workload adoption framework using scheduler customization and workload migration decisions, pp 41–45

  28. Kemp R, Palmer N, Kielmann T, Bal H (2012) Cuckoo: a computation offloading framework for smartphones, p 10

  29. Khalaj A, Lutfiyya H (2013) Handoff between proxies in the proxy-based mobile computing system. In: 2013 International Conference on MOBILe Wireless MiddleWARE, operating systems, and applications, pp 10–18

  30. Khan A, Othman M, Madani S, Khan S (2014) A survey of mobile cloud computing application models. Commun Surv Tutor IEEE 16(1):393–413

    Article  Google Scholar 

  31. Khan AN, Mat Kiah ML, Madani SA, Khan AUR, Ali M (2013) Enhanced dynamic credential generation scheme for protection of user identity in mobile-cloud computing. J Supercomput 66(3):1687–1706

    Article  Google Scholar 

  32. Kitchenham BA, Budgen D, Brereton OP (2011) Using mapping studies as the basis for further research a participant-observer case study. Inf Softw Technol 53(6):638–651

    Article  Google Scholar 

  33. Kosta S, Aucinas A, Mortier R (2012) ThinkAir: dynamic resource allocation and parallel execution in the cloud for mobile code offloading. In: 2012 Proc. IEEE INFOCOM, pp 945–953. IEEE

  34. Kovachev D, Yu T, Klamma R (2012) Adaptive computation offloading from mobile devices into the cloud. In: 2012 IEEE 10th International Symposium on Parallel and Distributed Processing with Applications, pp 784–791

  35. Kumar K, Liu J, Lu YH, Bhargava B (2013) A survey of computation offl.g for mobile systems. Mob Netw Appl 18(1):129–140

  36. Kwon Y-W, Tilevich E (2012) Energy-efficient and fault-tolerant distributed mobile execution. In: 2012 IEEE 32nd International Conference on Distributed Computing Systems, pp 586–595

  37. Kwon Y-W, Tilevich E (2013) Reducing the Energy Consumption of Mobile Applications Behind the Scenes. 2013 IEEE International Conference on Software Maintenance, pages 170–179, Sept. 2013

  38. Lim K-H, Lee B-D (2014) History-based dynamic estimation of energy consumption for mobile applications. In: 16th International Conference on Advanced Communication Technology, pp 714–718

  39. Liu Q, Jian X, Hu J, Zhao H, Zhang S (2009) An optimized solution for mobile environment using mobile cloud computing. In: Wireless communications, networking and mobile computing. WiCom ’09. 5th International Conference on, pp 1–5

  40. Montesi M, Lago P (2008) Software engineering article types: an analysis of the literature. J Syst Softw 81(10):1694–1714 (selected papers from the 30th Annual International Computer Software and Applications Conference (COMPSAC), Chicago, 2006)

  41. Muraleedharan R (2012) Cloud-vision: real-time face recognition using a mobile-cloudlet-cloud acceleration architecture. In: Proc. of the 2012 IEEE Symposium on Computers and Communications (ISCC), ISCC ’12, pp 59–66. IEEE Computer Society, Washington, DC

  42. Murugesan S, Venkatakrishnan B (2005) Addressing the challenges of web applications on mobile handheld devices. In: Mobile business. ICMB 2005. International Conference on, pp 199–205

  43. Namboodiri V, Ghose T (2012) To cloud or not to cloud: a mobile device perspective on energy consumption of applications. In: World of Wireless, Mobile and Multimedia Networks (WoWMoM), IEEE International Symposium on a, pp 1–9

  44. Niyato D (2012) A dynamic offloading algorithm for mobile computing. IEEE Trans Wirel Commun 11(6):1991–1995

    Article  Google Scholar 

  45. Petersen K, Feldt R, Mujtaba S, Mattsson M (2008) Systematic mapping studies in software engineering. In: Proc. of the 12th Conf. on Evaluation and Assessment in Software Engineering, EASE’08, pp 68–77. British Computer Society, Swinton

  46. Pu L, Xu J, Jin X, Zhang J (2013) SmartVirtCloud: virtual cloud assisted application offloading execution at mobile devices’ discretion. In: 2013 IEEE Wireless Communications and Networking Conference (WCNC), pp 4398–4403

  47. Ra M-R, Sheth A, Mummert L, Pillai P, Wetherall D, Govindan R (2011) Odessa: enabling interactive perception applications on mobile devices. In: Proc. of the 9th International Conference on Mobile Systems, Applications, and Services, MobiSys ’11, pp 43–56. ACM, New York

  48. Ravi A, Peddoju SK (2014) Mobility managed energy efficient Android mobile devices using cloudlet. In: Proc. of the 2014 IEEE Students’ Technology Symposium, pp 402–407

  49. Rodriguez JM, Mateos C, Zunino A (2014) Energy-efficient job stealing for cpu-intensive processing in mobile devices. Computing 96(2):87–117

    Article  MATH  Google Scholar 

  50. Saab SA, Chehab A, Kayssi A (2013) Energy efficiency in mobile cloud computing: total offloading selectively works. Does selective offloading totally work? In: 2013 4th Annual International Conference on Energy Aware Computing Systems and Applications (ICEAC), pp 164–168

  51. Saarinen A, Siekkinen M, Xiao Y, Nurminen JK, Kemppainen M, Hui P (2012) Can offloading save energy for popular apps? In: Proceedings of the seventh ACM international workshop on Mobility in the evolving internet architecture, pp 3–10. ACM

  52. Shoukry O, Fayek M (2013) Evolutionary scheduling for mobile content pre-fetching. In: Dediu A-H, Martin-Vide C, Truthe B, Vega-Rodriguez M (eds) Theory and practice of natural computing. Lecture notes in computer science, vol 8273. Springer, Berlin, Heidelberg, pp 228–239

    Chapter  Google Scholar 

  53. Silva FA, Silveira P, Garcia V, Assad R, Trinta F (2012) Accounting models for cloud computing: a systematic mapping study. In: International Conference in Grid Computing and Applications Proc. (GCA), pp 3–9

  54. Chinese/Hong Kong border automated with biometrics (2007) Biometric technology today 15(5):3

  55. Vallina-Rodriguez N, Crowcroft J (2013) Energy management techniques in modern mobile handsets. Commun Surv Tutor IEEE 15(1):179–198

    Article  Google Scholar 

  56. Verbelen T, Hens R, Stevens T, De Turck F, Dhoedt B (2010) Adaptive online deployment for resource constrained mobile smart clients. In: Cai Y, Magedanz T, Li M, Xia J, Giannelli C (eds) Mobile wireless middleware, operating systems, and applications, vol 48. Springer, Berlin, Heidelberg, pp 115–128

    Chapter  Google Scholar 

  57. Verbelen T, Simoens P, De Turck F, Dhoedt B (2012) Adaptive application configuration and distribution in mobile cloudlet middleware. In: Mobile wireless middleware, operating systems, and applications, pp 178–191. Springer, New York

  58. Verbelen T, Stevens T, Simoens P, De Turck F, Dhoedt B (2011) Dynamic deployment and quality adaptation for mobile augmented reality applications. J Syst Softw 84(11):1871–1882

    Article  Google Scholar 

  59. Wang X, Liu X, Huang G, Liu Y (2013) Appmobicloud: improving mobile web applications by mobile-cloud convergence. In: Proceedings of the 5th Asia-Pacific Symposium on Internetware, Internetware ’13, pp 14:1–14:10. ACM, New York

  60. Wang Y-C, Donyanavard B, Cheng K-TT (2010) Energy-aware real-time face recognition system on mobile cpu-gpu platform. In: Trends and Topics in Computer Vision, pp 411–422. Springer, New York

  61. Xia F, Ding F, Li J, Kong X, Yang LT, Ma J (2014) Phone2cloud: exploiting computation offloading for energy saving on smartphones in mobile cloud computing. Inf Syst Front 16(1):95–111

    Article  Google Scholar 

  62. Xia F, Hsu C-H, Liu X, Liu H, Ding F, Zhang W (2015) The power of smartphones. Multimed Syst 21(1):87–101

    Article  Google Scholar 

  63. Yang S, Kwon Y, Cho Y, Yi H, Kwon D, Youn J, Paek Y (2013) Fast dynamic execution offloading for efficient mobile cloud computing. In: 2013 IEEE International Conference on Pervasive Computing and Communications (PerCom), pp 20–28

  64. Zhang W, Chen L, Liu X, Lu Q, Zhang P, Yang S (2014) An OSGi-based flexible and adaptive pervasive cloud infrastructure. Sci China Inf Sci 57(3):1–11

    Google Scholar 

  65. Zhang X, Kunjithapatham A, Jeong S, Gibbs S (2011) Towards an elastic application model for augmenting the computing capabilities of mobile devices with cloud computing. Mobile Netw Appl 16(3):270–284

    Article  Google Scholar 

  66. Zhang Y, Liu H, Jiao L, Fu X (2012) To offload or not to offload: an efficient code partition algorithm for mobile cloud computing. In: 2012 IEEE 1st International Conference on Cloud Networking (CLOUDNET), pp 80–86

  67. Zhou Y, Jiang X (2012) Dissecting android malware: characterization and evolution. In: Security and Privacy (SP), 2012 IEEE Symposium on, pp 95–109

  68. Zhu H, Huang C, Yan J (2013) Vulnerability evaluation for securely offloading mobile apps in the cloud. In: 2013 IEEE 2nd Int. Conf. on Cloud Networking (CloudNet), pp 108–116

Download references

Author information

Authors and Affiliations

  1. Informatics Center, Federal University of Pernambuco, Road Jorn. Aníbal Fernandes, Cidade Universitária, Recife, PE, 50740-560, Brazil

    Francisco Airton Silva, Germano Zaicaner, Eder Quesado, Matheus Dornelas, Bruno Silva & Paulo Maciel

Authors
  1. Francisco Airton Silva
    View author publications

    You can also search for this author inPubMed Google Scholar

  2. Germano Zaicaner
    View author publications

    You can also search for this author inPubMed Google Scholar

  3. Eder Quesado
    View author publications

    You can also search for this author inPubMed Google Scholar

  4. Matheus Dornelas
    View author publications

    You can also search for this author inPubMed Google Scholar

  5. Bruno Silva
    View author publications

    You can also search for this author inPubMed Google Scholar

  6. Paulo Maciel
    View author publications

    You can also search for this author inPubMed Google Scholar

Corresponding author

Correspondence to Francisco Airton Silva.

Appendix

Appendix

See Tables 2, 3, 4 and 5.

Table 2 List of benchmark’s categories and the respective papers that used them
Full size table
Table 3 List of metrics and the respective papers that evaluated them
Full size table
Table 4 Platforms evaluated in this paper with the respective papers that used them
Full size table
Table 5 List of applications that were used by the MCC research reported in this paper and available to download
Full size table

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Silva, F.A., Zaicaner, G., Quesado, E. et al. Benchmark applications used in mobile cloud computing research: a systematic mapping study. J Supercomput 72, 1431–1452 (2016). https://doi.org/10.1007/s11227-016-1674-2

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11227-016-1674-2

Keywords