Skip to main content

Advertisement

SpringerLink
Log in

Efficient elicitation of software configurations using crowd preferences and domain knowledge

  • Published:
Automated Software Engineering Aims and scope Submit manuscript

Abstract

As software systems grow in size and complexity, the process of configuring them to meet individual needs becomes more and more challenging. Users, especially those that are new to a system, are faced with an ever increasing number of configuration possibilities, making the task of choosing the right one more and more daunting. However, users are rarely alone in using a software system. Crowds of other users or the designers themselves can provide with examples and rules as to what constitutes a meaningful configuration. We introduce a technique for designing optimal interactive configuration elicitation dialogs, aimed at utilizing crowd and expert information to reduce the amount of manual configuration effort. A repository of existing user configurations supplies us with information about popular ways to complete an existing partial configuration. Designers augment this information with their own constraints. A Markov decision process (MDP) model is then created to encode configuration elicitation dialogs that maximize the automatic configuration decisions based on the crowd and the designers’ information. A genetic algorithm is employed to solve the MDP when problem sizes prevent use of common exact techniques. In our evaluation with various configuration models we show that the technique is feasible, saves configuration effort and scales for real problem sizes of a few hundreds of features.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9

Similar content being viewed by others

Notes

  1. A prototypical implementation of the techniques described in this section, which are also used in the evaluation (Sect. 5) can be found here: https://github.com/yasserglez/configurator.

  2. http://www.facebook.com.

  3. See Facebook’s Q2 2017 Earnings Slides at: https://investor.fb.com/financials/?section=secfilings.

  4. http://www.cs.waikato.ac.nz/ml/weka/index.html.

  5. https://www.mathworks.com/matlabcentral/fileexchange/25786-markov-decision-processes--mdp--toolbox.

  6. https://archive.ics.uci.edu/ml/datasets/mushroom.

  7. http://www.splot-research.org.

  8. http://www.jku.at/isse/content/e139529/e126342/e126343.

  9. https://support.microsoft.com/en-ca/help/17479/.

References

  • Acher, M., Baudry, B., Heymans, P., Cleve, A., Hainaut, J.-L.: Support for reverse engineering and maintaining feature models. In: Proceedings of the 7th International Workshop on Variability Modelling of Software-Intensive Systems (VaMoS’13), pp. 20:1–20:8, Pisa (2013)

  • Acquisti, A., Gross, R.: Imagined Communities: Awareness, Information Sharing, and Privacy on the Facebook. Privacy Enhancing Technologies. Lecture Notes in Computer Science, vol. 4258, pp. 36–58. Springer, Berlin (2006)

    Google Scholar 

  • Adomavicius, G., Tuzhilin, A.: Toward the next generation of recommender systems: a survey of the state-of-the-art and possible extensions. IEEE Trans. Knowl. Data Eng. 17(6), 734–749 (2005)

    Article  Google Scholar 

  • Agrawal, R., Imieliński, T., Swami, A.: Mining association rules between sets of items in large databases. In: Proceedings of the 1993 ACM SIGMOD International Conference on Management of Data, pp. 207–216, Washington, DC (1993)

  • Bachmann, F., Goedicke, M., Leite, J., Nord, R., Pohl, K., Ramesh, B., Vilbig, A.: A meta-model for representing variability in product family development. In: Proceedings of the 5th International Workshop on Software Product-Family Engineering (PFE 2003)—Revised Papers, pp. 66–80 (2004)

  • Baghel, M., Agrawal, S., Silakari, S.: Survey of metaheuristic algorithms for combinatorial optimization. Int. J. Comput. Appl. 58(19), 21–31 (2012)

    Google Scholar 

  • Batory, D.: Feature models, grammars, and propositional formulas. Softw. Prod. Lines 3714, 7–20 (2005)

    Article  Google Scholar 

  • Bellman, R.E.: Dynamic Programming. Princeton University Press, Princeton (1957)

    MATH  Google Scholar 

  • Benavides, D., Trinidad, P., Ruiz-Cortés, A.: Automated reasoning on feature models. In: Proceedings of the Advanced Information Systems Engineering (CAiSE’05), pp. 491–503, Porto (2005)

  • Bierwirth, C., Mattfeld, D.C., Kopfer, H.: On Permutation Representations for Scheduling Problems. Parallel Problem Solving from Nature—PPSN IV, pp. 310–318. Springer, Berlin (1996)

    Google Scholar 

  • Blum, C., Roli, A.: Metaheuristics in combinatorial optimization: overview and conceptual comparison. ACM Comput. Surv. 35(3), 268–308 (2003)

    Article  Google Scholar 

  • Boutilier, C., Dean, T., Hanks, S.: Decision-theoretic planning: structural assumptions and computational leverage. J. Artif. Intell. Res. 11(1), 1–94 (1999)

    Article  MathSciNet  MATH  Google Scholar 

  • Boyd, D.M., Ellison, N.B.: Social network sites: definition, history, and scholarship. J. Comput. Med. Commun. 13(1), 210–230 (2007)

    Article  Google Scholar 

  • Boyd, D.M., Hargittai, E.: Facebook privacy settings: who cares? First Monday 15(8), 1 (2010)

    Article  Google Scholar 

  • Choudhary, A.K., Harding, J.A., Tiwari, M.K.: Data mining in manufacturing: a review based on the kind of knowledge. J. Intell. Manuf. 20(5), 501–521 (2009)

    Article  Google Scholar 

  • Consens, M.P., Barbosa, D., Teisanu, A., Mignet, L.: Goals and benchmarks for autonomic configuration recommenders. In: Proceedings of the 2005 ACM SIGMOD International Conference on Management of Data, pp. 239–250, Baltimore (2005)

  • Czarnecki, K., Eisenecker, U.W.: Generative Programming—Methods, Tools, and Applications. Addison-Wesley, Reading (2000)

    Google Scholar 

  • Czarnecki, K., Helsen, S., Ulrich, E.: Formalizing cardinality-based feature models and their specialization. Softw. Process. Improv. Pract. 10, 7–29 (2005)

    Article  Google Scholar 

  • Czarnecki, K., She, S., Wasowski, A.: Sample spaces and feature models: there and back again. In: Proceedings of the 12th International Software Product Line Conference (SPLC’08), pp. 22–31, Limerick (2008)

  • Czarnecki, K., Grünbacher, P., Rabiser, R., Schmid, K., Wąsowski, A.: Cool features and tough decisions: a comparison of variability modeling approaches. In: Proceedings of the 6th International Workshop on Variability Modeling of Software-Intensive Systems (VaMoS’12), pp. 173–182, Leipzig (2012)

  • Dardenne, A., van Lamsweerde, A., Fickas, S.: Goal-directed requirements acquisition. Sci. Comput. Program. 20(1–2), 3–50 (1993)

    Article  MATH  Google Scholar 

  • Debatin, B., Lovejoy, J.P., Horn, A.-K., Hughes, B.N.: Facebook and online privacy: attitudes, behaviors, and unintended consequences. J. Comput. Med. Commun. 15(1), 83–108 (2009)

    Article  Google Scholar 

  • Felfernig, A., Hotz, L., Bagley, C., Tiihonen, J.: Knowledge-Based Configuration: From Research to Business Cases. Morgan Kaufmann, Los Altos (2014)

    Google Scholar 

  • French, S.: Sequencing and Scheduling: An Introduction to the Mathematics of the Job-Shop. Wiley, New York (1982)

    MATH  Google Scholar 

  • Goldberg, D.E.: Genetic Algorithms in Search, Optimization, and Machine Learning. Addison-Wesley, Reading (1989)

    MATH  Google Scholar 

  • Gonzalez-Fernandez, Y.: Efficient calculation of optimal configuration processes. Master’s Thesis, York University, Ontario (2015)

  • Grefenstette, J.J., Gopal, R., Rosmaita, B., Van Gucht, D.: Genetic algorithms for the traveling salesman problem. In: Proceedings of the 1st International Conference on Genetic Algorithms and their Applications, pp. 160–165. Carnegie-Mellon University, Pittsburg (1985)

  • Gross, R., Acquisti, A.: Information revelation and privacy in online social networks. In: Proceedings of the 2005 ACM Workshop on Privacy in the Electronic Society, pp. 71–80, Alexandria (2005)

  • Guo, J., Czarnecki, K., Apel, S., Siegmund, N., Wasowski, A.: Variability-aware performance prediction: a statistical learning approach. In: Proceedings of the 28th IEEE/ACM International Conference on Automated Software Engineering (ASE’13), pp. 301–311, Silicon Valley (2013)

  • Hamidi, S.: Automating software customization via crowdsourcing using association rule mining and Markov decision processes. Master’s Thesis, York University, Ontario (2014)

  • Hamidi, S., Andritsos, P., Liaskos, S.: Constructing adaptive configuration dialogs using crowd data. In: Proceedings of the 29th ACM/IEEE International Conference on Automated Software Engineering (ASE’14), pp. 485–490, Västerås (2014)

  • Han, J., Pei, J., Yin, Y.: Mining frequent patterns without candidate generation. In: Proceedings of the 2000 ACM SIGMOD International Conference on Management of Data, pp. 1–12, Dallas (2000)

  • Hansen, T., Scheer, C., Gutenberg, J., Loos, P.: Product configurators in electronic commerce–extension of the configurator concept towards customer recommendation. In: Proceedings of the 2nd Interdisciplinary World Congress on Mass Customization and Personalization (MCP), Munich (2003)

  • Haslinger, E.N., Lopez-Herrejon, R.E., Egyed, A.: Reverse engineering feature models from programs’ feature sets. In: Proceedings of the 18th Working Conference on Reverse Engineering (WCRE’11), pp. 308–312, Limerick (2011)

  • Holland, J.H.: Adaptation in Natural and Artificial Systems: An Introductory Analysis with Applications to Biology, Control, and Artificial Intelligence. University of Michigan Press, Ann Arbor (1975)

    MATH  Google Scholar 

  • Jin, D., Cohen, M.B., Qu, X., Robinson, B.: Preffinder: getting the right preference in configurable software systems. In: Proceedings of the 29th ACM/IEEE International Conference on Automated Software Engineering (ASE’14), pp. 151–162, Västerås (2014)

  • Johnson, R.A., Wichern, D.W.: Applied Multivariate Statistical Analysis, 6th edn. Prentice Hall, Pearson (2007)

    MATH  Google Scholar 

  • Johnson, M., Egelman, S., Bellovin, S.M.: Facebook and privacy: it’s complicated. In: Proceedings of the 8th Symposium on Usable Privacy and Security (SOUPS’12), pp. 1–15, Washington, DC (2012)

  • Lampe, C., Ellison, N.B., Steinfield, C.: Changes in use and perception of Facebook. In: Proceedings of the 2008 ACM Conference on Computer Supported Cooperative Work (CSCW’08), pp. 721–730, San Diego (2008)

  • Lau, S.Q.: Domain analysis of e-commerce systems using feature-based model templates. Master’s Thesis, University of Waterloo, Ontario (2006)

  • Liaskos, S., Lapouchnian, A., Wang, Y., Yu, Y., Easterbrook, S.: Configuring common personal software: a requirements-driven approach. In: Proceedings of the 13th IEEE International Requirements Engineering Conference (RE’05), pp. 9–18, Paris (2005)

  • Liaskos, S., McIlraith, S.A., Sohrabi, S., Mylopoulos, J.: Representing and reasoning about preferences in requirements engineering. Requir. Eng. 16(3), 227–249 (2011)

    Article  Google Scholar 

  • Liaskos, S., Khan, S.M., Litoiu, M., Jungblut, M.D., Rogozhkin, V., Mylopoulos, J.: Behavioral adaptation of information systems through goal models. Inf. Syst. 37(8), 767–783 (2012)

    Article  Google Scholar 

  • Liu, Y., Gummadi, K.P., Krishnamurthy, B., Mislove, A.: Analyzing Facebook privacy settings: user expectations vs. reality. In: Proceedings of the 2011 ACM SIGCOMM Conference on Internet Measurement Conference (IMC’11), pp. 61–70, Berlin (2011)

  • Lopez-Herrejon, R.E., Galindo, J.A., Benavides, D., Segura, S., Egyed, A.: Reverse Engineering Feature Models with Evolutionary Algorithms: An Exploratory Study. Search Based Software Engineering, pp. 168–182. Springer, Berlin (2012)

    Google Scholar 

  • Mackay, W.E.: Triggers and barriers to customizing software. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI’91), pp. 153–160, New Orleans (1991)

  • Mackworth, A.K.: Consistency in networks of relations. Artif. Intell. 8, 99–118 (1977)

    Article  MATH  Google Scholar 

  • Mahmood, T., Ricci, F.: Adapting the interaction state model in conversational recommender systems. In: Proceedings of the 10th International Conference on Electronic Commerce (ICEC’08), pp. 33:1–33:10, Innsbruck (2008)

  • Mazo, R., Dumitrescu, C., Salinesi, C., Diaz, D.: Recommendation heuristics for improving product line configuration processes. In: Recommendation Systems in Software Engineering, pp. 511–537 (2014)

  • Mei, H., Zhang, W., Gu, F.: A feature oriented approach to modeling and reusing requirements of software product lines. In: Proceedings of the 27th Annual International Computer Software and Applications Conference (CASCON’17), pp. 250–256, Toronto (2003)

  • Mendonca, M., Branco, M., Cowan, D.: SPLOT: Software product lines online tools. In: Proceedings of the 24th ACM SIGPLAN Conference Companion on Object Oriented Programming Systems Languages and Applications (OOPSLA’09), pp. 761–762, Orlando (2009)

  • Millo, J.-V., Ramesh, S., Krishna, S.N., Narwane, G.K.: Compositional verification of software product lines. In: Integrated Formal Methods, Volume 7940 of Lecture Notes in Computer Science, pp. 109–123. Springer, Berlin (2013)

  • Moriarty, D.E., Schultz, A.C., Grefenstette, J.J.: Evolutionary algorithms for reinforcement learning. J. Artif. Intell. Res. 11, 241–276 (1999)

    Article  MATH  Google Scholar 

  • Mylopoulos, J., Chung, L., Liao, S., Wang, H., Eric, Y.: Exploring alternatives during requirements analysis. IEEE Softw. 18(1), 92–96 (2001)

    Article  Google Scholar 

  • Nöhrer, A., Egyed, A.: Optimizing user guidance during decision-making. In: Proceedings of the 15th International Software Product Line Conference (SPLC’15), pp. 25–34, Munich (2011)

  • Nöhrer, A., Egyed, A.: C\(_{2}\)O configurator: a tool for guided decision-making. Autom. Softw. Eng. 20(2), 265–296 (2013)

    Article  Google Scholar 

  • Page, S.R., Johnsgard, T.J., Albert, U., Dennis Allen, C.: User customization of a word processor. In: Proceeding of the 1996 SIGCHI Conference on Human Factors in Computing Systems (CHI’96), pp. 340–346, Vancouver (1996)

  • Pereira, J.A., Schulze, S., Figueiredo, E., Saake, G.: \(N\)-dimensional tensor factorization for self-configuration of software product lines at runtime. In: Proceedings of the 22nd International Conference on Systems and Software Product Line (SPLC’18), pp. 87–97, Gothenburg (2018a)

  • Pereira, J.A., Schulze, S., Krieter, S., Ribeiro, M., Saake, G.: A context-aware recommender system for extended software product line configurations. In: Proceedings of the 12th International Workshop on Variability Modelling of Software-Intensive Systems (VAMOS 2018), pp. 97–104, Madrid (2018b)

  • Pereira, J.A., Matuszyk, P., Krieter, S., Spiliopoulou, M., Saake, G.: Personalized recommender systems for product-line configuration processes. Comput. Lang. Syst. Struct. 1, 1–21 (2018c)

    Google Scholar 

  • Potvin, J.-Y.: Genetic algorithms for the traveling salesman problem. Ann. Oper. Res. 63, 339–370 (1996)

    Article  MATH  Google Scholar 

  • Puterman, M.L.: Markov Decision Processes: Discrete Stochastic Dynamic Programming. Wiley, New York (1994)

    Book  MATH  Google Scholar 

  • Riedmiller, M.: Neural fitted Q iteration—first experiences with a data efficient neural reinforcement learning method. In: Proceedings of the 16th European Conference on Machine Learning, pp. 317–328, Porto (2005)

  • Sarkar, A., Guo, J., Siegmund, N., Apel, S., Czarnecki, K.: Cost-efficient sampling for performance prediction of configurable systems. In: Proceedings of the 30th IEEE/ACM International Conference on Automated Software Engineering (ASE’15), pp. 342–352, Lincoln (2015)

  • Sayyad, A.S., Menzies, T., Ammar, H.: On the value of user preferences in search-based software engineering: a case study in software product lines. In: Proceedings of the 2013 International Conference on Software Engineering (ICSE’13), pp. 492–501. IEEE Press, San Francisco (2013)

  • Siegmund, N., Rosenmüller, M., Kuhlemann, M., Kästner, C., Apel, S., Saake, G.: SPL conqueror: toward optimization of non-functional properties in software product lines. Softw. Qual. J. 20(3), 487–517 (2012)

    Article  Google Scholar 

  • Siegmund, N., Grebhahn, A., Apel, S., Kästner, C.: Performance-influence models for highly configurable systems. In: Proceedings of the 2015 10th Joint Meeting on Foundations of Software Engineering (ESEC/FSE 2015), pp. 284–294, Bergamo (2015)

  • Soltani, S., Asadi, M., Hatala, M., Gas̆ević, G., Bagheri, E.: Automated planning for feature model configuration based on stakeholders’ business concerns. In: Proceedings of the 26th IEEE/ACM International Conference on Automated Software Engineering (ASE’11), pp. 536–539, Lawrence (2011)

  • Strater, K., Lipford, H.R.: Strategies and struggles with privacy in an online social networking community. In: Proceedings of the 22nd British HCI Group Annual Conference on People and Computers: Culture, Creativity, Interaction (BCS-HCI’08), pp. 111–119, Liverpool (2008)

  • Sutton, R.S., Barto, A.G.: Reinforcement Learning: An Introduction. MIT Press, Cambridge (1998)

    Google Scholar 

  • Tsang, E.: Foundations of Constraint Satisfaction. Academic, New York (1993)

    Google Scholar 

  • von der Maßen, T., Lichter, H.: Determining the variation degree of feature models. In: Proceedings of the 9th International Software Product Line Conference (SPLC’05), Rennes, Volume 3714 of Lecture Notes in Computer Science, pp. 82–88. Springer, Berlin (2005)

  • Whiteson, S.: Evolutionary computation for reinforcement learning. In: Wiering, M., van Otterlo, M. (eds.) Reinforcement Learning: State-of-the-Art, Volume 12 of Adaptation, Learning, and Optimization. Springer, Berlin (2012)

    Google Scholar 

  • Wilcoxon, F.: Individual comparisons by ranking methods. Biomet. Bull. 1(6), 80–83 (1945)

    Article  Google Scholar 

  • Xu, T., Jin, L., Fan, X., Zhou, Y., Pasupathy, S., Talwadker, R.: Hey, you have given me too many knobs! understanding and dealing with over-designed configuration in system software. In: Proceedings of the 2015 10th Joint Meeting on Foundations of Software Engineering (ESEC/FSE 2015), pp. 307–319. ACM, Bergamo (2015)

  • Zhang, B., Becker, M.: Mining complex feature correlations from software product line configurations. In: Proceedings of the 7th International Workshop on Variability Modelling of Software-Intensive Systems (VaMoS’13), pp. 19:1–19:7. ACM, Pisa (2013)

  • Zhang, G., Ye, H., Lin, Y.: Using knowledge-based systems to manage quality attributes in software product lines. In: Proceedings of the 15th International Software Product Line Conference (SPLC’11), pp. 32:1–32:7. ACM, Munich (2011)

Download references

Author information

Authors and Affiliations

  1. School of Information Technology, York University, Toronto, Canada

    Yasser Gonzalez-Fernandez, Saeideh Hamidi, Stephen Chen & Sotirios Liaskos

Authors
  1. Yasser Gonzalez-Fernandez
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Saeideh Hamidi
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Stephen Chen
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Sotirios Liaskos
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Sotirios Liaskos.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Gonzalez-Fernandez, Y., Hamidi, S., Chen, S. et al. Efficient elicitation of software configurations using crowd preferences and domain knowledge. Autom Softw Eng 26, 87–123 (2019). https://doi.org/10.1007/s10515-018-0247-4

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10515-018-0247-4

Keywords

Search

Navigation