Skip to main content
SpringerLink
Log in

Open issues in model transformations for multimodal applications

  • Original Paper
  • Published:
Journal on Multimodal User Interfaces Aims and scope Submit manuscript

Abstract

Multimodal human–computer interaction refers to the interaction with the virtual and physical environment through natural modes of communication. Multimodal input user interfaces have significant role in different domains, for example industrial plants or hospitals, also they have implications for accessibility. To develop multimodal applications in formally rigorous settings, software developer teams may use tools or a software development kit to increase the efficiency and the quality of the resulted software artifacts. Such a technique is performing the design with software modeling and applying model transformations to generate well-defined components of the software. Furthermore, representation-bridging communication is a discipline of cognitive infocommunications, where the sensory information transferred to the receiver entity is filtered and/or converted. Whenever such approaches are used, the challenges associated with the modeling of information requirements, user capabilities and cross-model interactions are compounded and further increase the need for formal design and verification tools. Applying model transformations is a way to support this activity. Communication-intensive solutions often require complex methods, i.e. significant model transformation efforts between the different representations. Important semantic information should be preserved and not misinterpreted in a complex model transformations. Therefore, methods are required to verify that the semantics used during the application generation and analysis are indeed preserved across the transformation. As a case in point, such a model transformation could yield embedded code for a given type of electronic driver assistant system based on a high-level characterizations of the information to be transferred and the driver’s cognitive capabilities. Later, a multimodal interactions expert could easily modify those characterizations on demand, and regenerate a modified version of the software without having to know about the low-level details of the embedded platform. This paper provides a strong motivation regarding the necessity of methods to support verification and validation of model transformations supporting multimodal application development and cognitive infocommunications. As the main result of the paper, we compile a list of open issues in the field of verification/validation of model transformations, and link those issues to the development of multimodal interfaces. Through its discussions, the paper makes the point that the design practices behind multimodal interfaces could strongly benefit from the use of formal modeling techniques in general, and model transformation approaches in particular.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1

Similar content being viewed by others

References

  1. AGG: The attributed graph grammar system website. http://tfs.cs.tu-berlin.de/agg

  2. Akehurst D, Kent S (2002) A relational approach to defining transformations in a metamodel. In: UML 2002—the unified modeling language, 5th international conference, vol 2460. LNCS, Springer, Dresden, Germany, pp 243–258

  3. Amrani M, Dingel J, Lambers L, Lcio L, Salay R, Selim G, Syriani E, Wimmer M (2012) Towards a model transformation intent catalog. In: Proceedings of the first workshop on the analysis of model transformations (AMT ’12). ACM, New York, pp 3-8. doi:10.1145/2432497.2432499

  4. AToM3: A tool for multi-paradigm, multi-formalism and meta-modeling website. http://atom3.cs.mcgill.ca

  5. Asztalos M, Lengyel L, Levendovszky T (2013) Formal specification and analysis of functional properties of graph rewriting-based model transformation. Softw Test Verif Reliab 23(5):405–435

    Article  Google Scholar 

  6. Anastasakis K, Bordbar B, Küster JM (2007) Analysis of model transformations via alloy. In: Workshop MoDeVVA, vol 07, pp 47–56

  7. Anastasakis K, Bordbar B, Georg G, Ray I (2007) UML2Alloy: a challenging model transformation. In: Proceedings of the MoDELS07, vol. 4735. LNCS, Springer, Berlin, pp 430–450

  8. Assmann U, Ludwig A (2000) Aspect weaving by graph rewriting, generative componentbased software engineering. In: Lecture Notes in Computer Science 1799. Springer, Berlin

  9. Assmann U (1996) How to uniformly specify program analysis and transformation with graph rewrite systems. In: Proceedings of the 6 international conference on compiler construction (CC) ’96, vol 1060. LNCS, Springer, Berlin

  10. Baldan P, Knig B (2002) Approximating the behaviour of graph transformation systems. In: Proceedings of the ICGT 2002, first international conference on graph transformation. Springer, Berlin, pp 14–29

  11. Baldan P, Knig B, Rensink A (2005) Graph grammar verification through abstraction. In: Graph transforamtion and process algebras for modeling distributed and mobile systems. Dagstuhl seminar, vol 04241

  12. Baranyi P, Csapo A (2010) Cognitive infocommunications: CogInfoCom. In: 11th IEEE international symposium on computational intelligence and informatics, Budapest

  13. Baranyi P, Csapo A (2012) Definition and synergies of cognitive infocommunications. Acta Polytechnica Hungarica 9(1):67–83

    Google Scholar 

  14. Barbosa P, Ramalho F, Figueiredo J, Junior A, Costa A, Gomes L (2009) Checking semantics equivalence of MDA transformations in concurrent systems. J. Univ. Comp. Sci. 15(11):2196–2224

    Google Scholar 

  15. Biermann E, Ermel C, Taentzer G (2011) Formal foundation of consistent EMF model transformations by algebraic graph transformation, software and systems modeling (SoSyM). Springer, Berlin

  16. Bisztray D, Heckel R, Ehrig H (2004) Verification of architectural refactorings by rule extraction. In: Fundamental approaches to software engineering, vol 4961. LNCS, Springer, Berlin, pp 347–361

  17. Blostein D, Fahmy H, Grbavec A (1996) Issues in the practical use of graph rewriting. In: Proceedings of the 5th international workshop on graph grammars and their application to computer science, vol 1073, Williamsburg, USA, LNCS, Springer, Berlin, pp 38–55

  18. Braun P, Marschall F (2003) BOTL—the bidirectional object-oriented transformation language. Fakultat fur Informatik, Technische Universitat München, Technical report TUM-I0307

  19. Brucker AD, Wolff B (2006) The HOL-OCL book. Technical report, vol 525. ETH Zurich, Zurich

  20. Cabot J, Clariso R, Riera D (2008) Verification of UML/OCL class diagrams using constraint programming. In: MoDeVVa 2008, ICST workshop, pp 73–80

  21. Cabot J, Clariso R, Guerra E, de Lara J (2010) V&V of declarative model-to-model transformations through invariants. J. Syst. Softw. 83(2):283–302

    Article  Google Scholar 

  22. Czarnecki K, Helsen S (2006) Feature-based survey of model transformation approaches. IBM Syst. J. 45(3):621–646

    Article  Google Scholar 

  23. de Lara J, Taentzer G (2004) Automated model transformation and its validation with AToM3 and AGG. In: Diagrammatic representation and inference, lecture notes in artificial intelligence, vol 2980. Springer, Berlin, pp 182–198

  24. de Lara J, Vangheluwe H, Alfonseca M (2004) Metamodelling and graph grammars for multi-paradigm modelling in AToM3. J Softw Syst Model 3(3):194–209

    Google Scholar 

  25. Dotti FL, Foss L, Ribeiro L, dos Santos OM (2008) Verification of object-based distributed systems. In: Proceedings of the 6th international conference on formal methods for open object-based distributed systems, pp 261–275

  26. Ehrig H, Engels G, Kreowski H-J, Rozenberg G (eds) (1999) Handbook on graph grammars and computing by graph transformation: application, languages and tools, vol 2. World Scientific, Singapore

  27. Fujaba Tool Suite website. http://www.fujaba.de/

  28. Giese H, Glesner S, Leitner J, Schafer W, Wagner R (2006) Towards verified model transformations. In: ModeVVa06

  29. Gorp vP, Stenten H, Mens T, Demeyer S (2003) Towards automating source-consistent UML refactorings. In: UML 2003—the unified modeling language. modeling languages and applications, 6th international conference, San Francisco, USA, vol 2863. LNCS, Springer, Berlin, pp 144–158

  30. GReAT: graph rewriting and transformation website. http://www.isis.vanderbilt.edu/tools/GReAT

  31. Guerra E, de Lara J (2007) Event-driven grammars: relating abstract and concrete levels of visual languages. SoSym 6:317–347

    Google Scholar 

  32. Habel A, Heckel R, Taentzer G (1996) Graph grammars with negative application conditions. Fundamenta Informaticae 26:287–313

    MATH  MathSciNet  Google Scholar 

  33. Heckel R, Küster JM, Taentzer G (2002) Towards automatic translation of UML models into semantic domains. In: Proceedings of the appligraph workshop on applied graph transformation, pp 11–22

  34. Heckel R (1998) Compositional verification of reactive systems specified by graph transformation. In: FASE, pp 138–153

  35. Holzmann GJ (1997) The model checker SPIN. Softw Eng 23(5):279–295

    Article  MathSciNet  Google Scholar 

  36. Hulsbusch M, Konig B, Rensink A, Semenyak M, Soltenborn C, Wehrheim H (2010) Showing full semantics preservation in model transformation—a comparison of techniques. In: Integrated Formal Methods, vol 6396. Springer, LNCS, Berlin, pp 183–198

  37. Küster JM, Heckel R, Engels G (2003) Defining and validating transformations of UML models. In: IEEE symposium on human centric computing languages and environments, Auckland, New Zealand, pp 145–152

  38. Küster JM (2006) Definition and validation of model transformations. Softw Syst Model 5(3):233–259

    Article  Google Scholar 

  39. Lengyel L (2006) Online validation of visual model transformations. PhD thesis, Budapest University of Technology and Economics, Department of Automation and Applied Informatics

  40. Mens T, Demeyer S, Janssens D (2002) Formalising behaviour preserving program transformations. In: Proceedings of the first international conference on graph transformation. Springer, Berlin, London, pp 286–301

  41. Mens T, Tourwe T (2004) A survey of software refactoring. IEEE Trans Softw Eng 30(2):126–139

    Article  Google Scholar 

  42. Mens T, Gorp vP (2006) A taxonomy of model transformation, electronic notes in theoretical computer science, vol 152. In: Proceedings of the international workshop on graph and model transformation (GraMoT 2005), pp 125–142

  43. Narayanan A, Karsai G (2008) Towards verifying model transformations. ENTCS 211:191–200

    Google Scholar 

  44. OMG Model-Driven Architecture (MDA) Specification, OMG document ormsc/01-07-01 (2001). http://www.omg.org/

  45. Plump D (1998) Termination of graph rewriting is undecidable. In: Fundam. Inf., vol 33, issue 2. Amsterdam. IOS Press, The Netherlands, pp 201–209

  46. Pratt TW (1971) Pair grammars, graph languages and string-to-graph translations. J Comput Syst Sci 5:560–595

    Article  MATH  MathSciNet  Google Scholar 

  47. Rensink A, Schmidt A, Varró D (2004) Model checking graph transformations: a comparison of two approaches. In: Proceedings of the ICGT 2004: second international conference on graph transformation, vol 3256. LNCS, Springer, Rome, pp 226–241

  48. Rozenberg G (ed) (1997) Handbook on graph grammars and computing by graph transformation: foundations, vol 1. World Scientific, Singapore

  49. Schatz B (2010) Verification of model transformations. vol 29. ECEASST, Paphos, Cyprus, pp 129–142

  50. Schürr A (1994) Specification of graph translators with triple graph grammars. In: Proceedings of the WG94 international workshop on graph-theoretic concepts in computer science, vol 903. LNCS, Springer, Berlin, pp 151–163

  51. Straeten RVD, Mens T, Simmonds J, Jonckers V (2003) Using description logic to maintain consistency between UML models. In: Proceedings of the UML03, vol 2863. LNCS, Springer, Berlin, pp 326–340

  52. Taentzer G, Ehrig K, Guerra E, de Lara J, Lengyel L, Levendovszky T, Prange U, Varró D, Varró-Gyapay SZ (2005) Model transformation by graph transformation: a comparative study. In: ACM/IEEE 8th international conference on model driven engineering languages and systems, Montego Bay, Jamaica

  53. Varró D, Pataricza A (2003) Automated formal verification of model transformations. In: Proceedings of theUML03 workshop, technical report, pp 63–78

  54. VIATRA2 (VIsual Automated model TRAnsformations) framework website. http://eclipse.org/gmt/VIATRA2

  55. VMTS: Visual Modeling and Transformation System website. http://www.aut.bme.hu/vmts

Download references

Acknowledgments

This work was partially supported by the TÁMOP-4.2.1.D-15/1/KONV-2015-0008 project. This paper was supported by the János Bolyai Research Scholarship of the Hungarian Academy of Sciences.

Author information

Authors and Affiliations

  1. Department of Automation and Applied Informatics, Budapest University of Technology and Economics, Magyar Tudósok körútja 2, 1117, Budapest, Hungary

    László Lengyel & Hassan Charaf

Authors
  1. László Lengyel
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Hassan Charaf
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to László Lengyel.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Lengyel, L., Charaf, H. Open issues in model transformations for multimodal applications. J Multimodal User Interfaces 9, 377–385 (2015). https://doi.org/10.1007/s12193-015-0192-5

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s12193-015-0192-5

Keywords

Search

Navigation