skip to main content
10.1145/2642937.2642939acmconferencesArticle/Chapter ViewAbstractPublication PagesaseConference Proceedingsconference-collections
research-article

Automated variability analysis and testing of an E-commerce site.: an experience report

Published: 15 September 2014 Publication History

Abstract

In this paper, we report on our experience on the development of La Hilandera, an e-commerce site selling haberdashery products and craft supplies in Europe. The store has a huge input space where customers can place almost three millions of different orders which made testing an extremely difficult task. To address the challenge, we explored the applicability of some of the practices for variability management in software product lines. First, we used a feature model to represent the store input space which provided us with a variability view easy to understand, share and discuss with all the stakeholders. Second, we used techniques for the automated analysis of feature models for the detection and repair of inconsistent and missing configuration settings. Finally, we used test selection and prioritization techniques for the generation of a manageable and effective set of test cases. Our findings, summarized in a set of lessons learnt, suggest that variability techniques could successfully address many of the challenges found when developing e-commerce sites.

References

[1]
D. Benavides, S. Segura, and A. Ruiz-Cort A c s. Automated analysis of feature models 20 years later: A literature review. Information Systems, 35(6):615--636, 2010.
[2]
T. Berger, S. She, R. Lotufo, A. Wasowski, and K. Czarnecki. A study of variability models and languages in the systems software domain. IEEE Transactions on Software Engineering, 39(12):1611--1640, Dec 2013.
[3]
L. Copeland. A Practitioner's Guide to Software Test Design. Artech House, Inc., Norwood, MA, USA, 2003.
[4]
K. Czarnecki and M. Antkiewicz. Mapping features to models: A template approach based on superimposed variants. In Proceedings of the 4th International Conference on Generative Programming and Component Engineering, GPCE'05, pages 422--437, Berlin, Heidelberg, 2005. Springer-Verlag.
[5]
Debian 7.0 wheezy released, May 2013. Accessed November 2013.
[6]
X. Devroey, G. Perrouin, M. Cordy, P.-Y. Schobbens, A. Legay, and P. Heymans. Towards statistical prioritization for software product lines testing. In Eighth International Workshop on Variability Modelling of Software-Intensive Systems, 2014.
[7]
F. Ensan, E. Bagheri, and D. Gasevic. Evolutionary search-based test generation for software product line feature models. In Conference on Advanced Information Systems Engineering (CAiSE'12), 2012.
[8]
FaMa Tool Suite. http://www.isa.us.es/fama/, accessed April 2014.
[9]
J. Ferrer, P. Krse, F. Chicano, and E. Alba. Evolutionary algorithm for prioritized pairwise test data generation. In Proceedings of the 14th annual conference on Genetic and evolutionary computation, 2012.
[10]
J. García-Galán, O. Rana, P. Trinidad, and A. R. Cortés. Migrating to the cloud: a software product line based analysis. In 3rd International Conference on Cloud Computing and Services Science (CLOSER'13), 2013.
[11]
M. Grindal, J. Offutt, and S. Andler. Combination testing strategies: a survey. Software Testing, Verification and Reliability, 15(3):167--199, 2005.
[12]
C. Henard, M. Papadakis, G. Perrouin, J. Klein, P. Heymans, and Y. L. Traon. Bypassing the combinatorial explosion: Using similarity to generate and prioritize t-wise test suites for large software product lines. Technical report, 2012.
[13]
C. Henard, M. Papadakis, G. Perrouin, J. Klein, and Y. Traon. Multi-objective test generation for software product lines. In Proceedings of the 17th International Software Product Line Conference, SPLC '13, pages 62--71, New York, NY, USA, 2013. ACM.
[14]
I do proyect. http://idoproyect.com/, accessed April 2014.
[15]
M. F. Johansen, O. Haugen, and F. Fleurey. Properties of realistic feature models make combinatorial testing of product lines feasible. In MODELS, 2011.
[16]
M. F. Johansen, O. Haugen, F. Fleurey, A. G. Eldegard, and T. Syversen. Generating better partial covering arrays by modeling weights on sub-product lines. In International Conference MODELS, 2012.
[17]
K. Kang, S. Cohen, J. Hess, W. Novak, and S. Peterson. Feature--Oriented Domain Analysis (FODA) Feasibility Study. Technical Report Carnegie Mellon University/SEI-90-TR-21, SEI, 1990.
[18]
B. P. Lamancha and M. P. Usaola. Testing product generation in software product lines using pairwise for feature coverage. In International conference on Testing Software and Systems, 2010.
[19]
S. Lau and K. Czarnecki. Domain analysis of e-commerce systems using feature-based model templates. Master's thesis, University of Waterloo, Waterloo, 2006.
[20]
Y.-F. Li, P. K. Das, and D. L. Dowe. Two decades of web application testing - a survey of recent advances. Information Systems, 43:20--54, 2014.
[21]
R. Lopez-Herrejon, F. Chicano, J. Ferrer, A. Egyed, and E. Alba. Multi-objective optimal test suite computation for software product line pairwise testing. In Proceedings of the 29th IEEE International Conference on Software Maintenance, 2013.
[22]
Magento. http://magento.com/, accessed April 2014.
[23]
D. Marijan, A. Gotlieb, S. Sen, and A. Hervieu. Practical pairwise testing for software product lines. In Proceedings of the 17th International Software Product Line Conference, SPLC '13, pages 227--235, New York, NY, USA, 2013. ACM.
[24]
M. Mendonca, M. Branco, and D. Cowan. S.P.L.O.T.: Software Product Lines Online Tools. In Companion to the 24th ACM SIGPLAN International Conference on Object-Oriented Programming, Systems, Languages, and Applications (OOPSLA), pages 761--762, Orlando, Florida, USA, October 2009. ACM.
[25]
G. J. Myers and C. Sandler. The Art of Software Testing. John Wiley & Sons, 2004.
[26]
H. Nguyen, C. Kästner, and T. Nguyen. Exploring variability-aware execution for testing plugin-based web applications. In Proceedings of the 36th International Conference on Software Engineering (ICSE), 6 2014.
[27]
G. Perrouin, S. Oster, S. Sen, J. Klein, B. Budry, and Y. le Traon. Pairwise testing for software product lines: comparison of two approaches. Software Quality Journal, 2011.
[28]
PHPUnit.http://phpunit.de/, accessed April 2014.
[29]
Prestashop. http://www.prestashop.com/, accessed April 2014.
[30]
A. B. Sánchez, S. Segura, and A. Ruiz-Cortés. A comparison of test case prioritization criteria for software product lines. In IEEE International Conference on Software Testing, Verification, and Validation, 2014.
[31]
A. Sayyad, T. Menzies, and H. Ammar. On the value of user preferences in search-based software engineering: A case study in software product lines. In Proceedings of the 2013 International Conference on Software Engineering, ICSE '13, pages 492--501, Piscataway, NJ, USA, 2013. IEEE Press.
[32]
Selenium. http://docs.seleniumhq.org/, accessed April 2014.
[33]
Tricotplus. www.tricotplus.com, accessed April 2014.
[34]
We are knitters. http://www.weareknitters.com, accessed April 2014.
[35]
S. Wang, A. Gotlieb, S. Ali, and M. Liaaen. Automated test case selection using feature model: An industrial case study. In Model-Driven Engineering Languages and Systems, volume 8107 of Lecture Notes in Computer Science, pages 237--253. Springer Berlin Heidelberg, 2013.

Cited By

View all

Index Terms

  1. Automated variability analysis and testing of an E-commerce site.: an experience report

    Recommendations

    Comments

    Information & Contributors

    Information

    Published In

    cover image ACM Conferences
    ASE '14: Proceedings of the 29th ACM/IEEE International Conference on Automated Software Engineering
    September 2014
    934 pages
    ISBN:9781450330138
    DOI:10.1145/2642937
    Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

    Sponsors

    Publisher

    Association for Computing Machinery

    New York, NY, United States

    Publication History

    Published: 15 September 2014

    Permissions

    Request permissions for this article.

    Check for updates

    Author Tags

    1. automated testing
    2. e-commerce
    3. experience report
    4. feature modelling
    5. variability

    Qualifiers

    • Research-article

    Funding Sources

    Conference

    ASE '14
    Sponsor:

    Acceptance Rates

    ASE '14 Paper Acceptance Rate 82 of 337 submissions, 24%;
    Overall Acceptance Rate 82 of 337 submissions, 24%

    Contributors

    Other Metrics

    Bibliometrics & Citations

    Bibliometrics

    Article Metrics

    • Downloads (Last 12 months)8
    • Downloads (Last 6 weeks)0
    Reflects downloads up to 20 Feb 2025

    Other Metrics

    Citations

    Cited By

    View all
    • (2024)Speculative computing for AAFM solutions in large-scale product configurationsScientific Reports10.1038/s41598-024-61647-614:1Online publication date: 16-May-2024
    • (2024)UVLHubJournal of Systems and Software10.1016/j.jss.2024.112150216:COnline publication date: 1-Oct-2024
    • (2023)Cloud Development and DeploymentOptimising the Software Development Process with Artificial Intelligence10.1007/978-981-19-9948-2_6(155-176)Online publication date: 20-Jul-2023
    • (2022)Exploring Functionality and Efficiency of Feature Model Product Configuration SolutionsIEEE Access10.1109/ACCESS.2022.323144910(134318-134332)Online publication date: 2022
    • (2021)A first prototype of a new repository for feature model exchange and knowledge sharingProceedings of the 25th ACM International Systems and Software Product Line Conference - Volume B10.1145/3461002.3473949(80-85)Online publication date: 6-Sep-2021
    • (2019)Understanding similarities and differences in software development practices across domainsProceedings of the 14th International Conference on Global Software Engineering10.1109/ICGSE.2019.00013(74-84)Online publication date: 25-May-2019
    • (2017)Statistical prioritization for software product line testingSoftware and Systems Modeling (SoSyM)10.1007/s10270-015-0479-816:1(153-171)Online publication date: 1-Feb-2017
    • (2017)Variability testing in the wildSoftware and Systems Modeling (SoSyM)10.1007/s10270-015-0459-z16:1(173-194)Online publication date: 1-Feb-2017

    View Options

    Login options

    View options

    PDF

    View or Download as a PDF file.

    PDF

    eReader

    View online with eReader.

    eReader

    Figures

    Tables

    Media

    Share

    Share

    Share this Publication link

    Share on social media