An Energy-Based Approach Towards Modeling of Mixed Reality Mechatronic Systems

Abstract

An environment where virtual worlds coexist with real worlds, mixed reality, has been extensively studied in the last decade. Milgram’s definition of the Reality-Virtuality Continuum [8] focuses on display technology such as head-mounted displays. However, people still feel the gap between mixed reality environments supported only by display technology and those closely integrating physical reality [10].We do not restrict mixed reality to a mixing of visual images or sounds, but include other phenomena (e.g., electric, mechanical, hydraulic, aerodynamic, thermodynamic). Mechatronic technology can be used to enhance physical usability in mixed reality. Bond graphs, originated by Paynter [2] and further developed by Karnopp [3], are domain independent graphical descriptions of the dynamic behavior of physical systems. Amerongen [6] emphasized that the energy-based approaches toward the design of mechatronic systems allow the construction of reusable and easily extendible models. Bruns [4] introduced a universal interface for mixed reality using bond graphs, called hyper-bond. An improved hyper-bond was presented by Yoo and Bruns [5]. A real energy dynamics can connect with a virtual energy dynamics via an improved hyper-bond interface providing relevant functionalities such as preservation of power balance, causality reasoning of a connection between real and virtual subsystems, stability, and quality. This paper presents a new methodology of mixed reality design, called mixed reality bond graph, in which the modeling methods of real energy dynamics, virtual energy dynamics, and hyper-bond interfaces can be combined seamlessly. A distributed mixed reality haptic ball manipulator, as an example using the mixed reality bond graphs, is demonstrated.