Jan Sinschek
Andreas Sewe
Mira Mezini
ABSTRACT
The performance evaluation of virtual machines is notoriously difficult. Therefore, experimental methodology has recently drawn attention, leading to proposals on how to choose benchmarks, interpret results, and detect measurement bias. But this latter task currently relies on the presence of anomalous measurement results, i.e., on outliers, to raise suspicion. We therefore propose the use of functional performance models to detect bias even when benchmark results might appear unsuspicious. Failure to validate the model indicates either bias or a need to refine the model.
- S. M. Blackburn, R. Garner, C. Hoffmann, A. M. Khang, K. S. McKinley, R. Bentzur, A. Diwan, D. Feinberg, D. Frampton, S. Z. Guyer, M. Hirzel, A. Hosking, M. Jump, H. Lee, J. E. B. Moss, B. Moss, A. Phansalkar, D. Stefanović, T. VanDrunen, D. von Dincklage, and B. Wiedermann. The DaCapo benchmarks: Java benchmarking development and analysis. In Proceedings of the 21st OOPSLA Conference, pages 169--190, 2006. Google Scholar
Digital Library
- S. M. Blackburn, K. S. McKinley, R. Garner, C. Hoffmann, A. M. Khan, R. Bentzur, A. Diwan, D. Feinberg, D. Frampton, S. Z. Guyer, M. Hirzel, A. Hosking, M. Jump, H. Lee, J. E. B. Moss, A. Phansalkar, D. Stefanovik, T. VanDrunen, D. von Dincklage, and B. Wiedermann. Wake up and smell the coffee: evaluation methodology for the 21st century. Communications of the ACM, 51(8):83--89, 2008. Google ScholarDigital Library
- L. Eeckhout, A. Georges, and K. De Bosschere. How Java programs interact with virtual machines at the microarchitectural level. In Proceedings of the 18th OOPSLA Conference, pages 169--186, 2003. Google ScholarDigital Library
- A. Georges. Three Pitfalls in Java Performance Evaluation. PhD thesis, Universiteit Gent, 2008.Google Scholar
- A. Georges, D. Buytaert, and L. Eeckhout. Statistically rigorous Java performance evaluation. In Proceedings of the 22nd OOPSLA Conference, pages 57--76, 2007. Google ScholarDigital Library
- D. Gu, C. Verbrugge, and E. M. Gagnon. Relative factors in performance analysis of Java virtual machines. In Proceedings of the 2nd VEE Conference, pages 111--121, 2006. Google ScholarDigital Library
- P. Kulkarni, M. Arnold, and M. Hind. Dynamic compilation: the benefits of early investing. In Proceedings of the 3rd VEE Conference, pages 94--104, 2007. Google ScholarDigital Library
- T. Mytkowicz, A. Diwan, M. Hauswirth, and P. F. Sweeney. Producing wrong data without doing anything obviously wrong! In Proceedings of the 14th ASPLOS Conference, pages 265--276, 2009. Google ScholarDigital Library
Index Terms
- VM performance evaluation with functional models: an optimist's outlook
Recommendations
Performance Evaluation of VM Placement Using Classical Bin Packing and Genetic Algorithm for Cloud Environment
In current era, the trend of cloud computing is increasing with every passing day due to one of its dominant service i.e. Infrastructure as a service IAAS, which virtualizes the hardware by creating multiple instances of VMs on single physical machine. ...
Pre-Copy and post-copy VM live migration for memory intensive applications
Euro-Par'12: Proceedings of the 18th international conference on Parallel processing workshopsVirtualization technology provides a means for server consolidation, reducing the number of physical servers required for running a given workload. Virtual Machine (VM) live migration facilitates the transfer of a running (VM) between physical hosts ...
Virtual Device Passthrough for High Speed VM Networking
ANCS '15: Proceedings of the Eleventh ACM/IEEE Symposium on Architectures for networking and communications systemsSupporting network I/O at high packet rates in virtual machines is fundamental for the deployment of Cloud data centers and Network Function Virtualization. Historically, SR-IOV and hardware passthrough were thought as the only viable solution to reduce ...
Comments