Abstract
In this article, we propose a middleware that supports a developer to build tabletop information display systems. We focus on an application that projects information close to a particular object on a table. To let a user focus on his/her primary task, the information is presented based on a current phase in a procedure, which we call procedure-awareness. Here, a current phase is recognized through the utilization of objects. The proposed middleware separates information from acquisition and presentation mechanisms: sensors for operational context recognition, sensors for object identification/tracking for “nearby presentation”, and projecting information near an object. A developer just needs to write a set of rules, i.e. application logic, that represent contents for particular operational contexts. Also, by implementing appropriate handlers, any content can be provided. We present the design and the implementation of the middleware and validate the expressiveness of application logics through prototype development.
![](http://media.springernature.com/m312/springer-static/image/art%3A10.1007%2Fs11042-011-0759-4/MediaObjects/11042_2011_759_Fig1_HTML.jpg)
![](http://media.springernature.com/m312/springer-static/image/art%3A10.1007%2Fs11042-011-0759-4/MediaObjects/11042_2011_759_Fig2_HTML.gif)
![](http://media.springernature.com/m312/springer-static/image/art%3A10.1007%2Fs11042-011-0759-4/MediaObjects/11042_2011_759_Fig3_HTML.gif)
![](http://media.springernature.com/m312/springer-static/image/art%3A10.1007%2Fs11042-011-0759-4/MediaObjects/11042_2011_759_Fig4_HTML.gif)
![](http://media.springernature.com/m312/springer-static/image/art%3A10.1007%2Fs11042-011-0759-4/MediaObjects/11042_2011_759_Fig5_HTML.gif)
![](http://media.springernature.com/m312/springer-static/image/art%3A10.1007%2Fs11042-011-0759-4/MediaObjects/11042_2011_759_Fig6_HTML.gif)
![](http://media.springernature.com/m312/springer-static/image/art%3A10.1007%2Fs11042-011-0759-4/MediaObjects/11042_2011_759_Fig7_HTML.gif)
![](http://media.springernature.com/m312/springer-static/image/art%3A10.1007%2Fs11042-011-0759-4/MediaObjects/11042_2011_759_Fig8_HTML.gif)
![](http://media.springernature.com/m312/springer-static/image/art%3A10.1007%2Fs11042-011-0759-4/MediaObjects/11042_2011_759_Fig9_HTML.gif)
![](http://media.springernature.com/m312/springer-static/image/art%3A10.1007%2Fs11042-011-0759-4/MediaObjects/11042_2011_759_Fig10_HTML.gif)
![](http://media.springernature.com/m312/springer-static/image/art%3A10.1007%2Fs11042-011-0759-4/MediaObjects/11042_2011_759_Fig11_HTML.jpg)
![](http://media.springernature.com/m312/springer-static/image/art%3A10.1007%2Fs11042-011-0759-4/MediaObjects/11042_2011_759_Fig12_HTML.gif)
![](http://media.springernature.com/m312/springer-static/image/art%3A10.1007%2Fs11042-011-0759-4/MediaObjects/11042_2011_759_Fig13_HTML.gif)
![](http://media.springernature.com/m312/springer-static/image/art%3A10.1007%2Fs11042-011-0759-4/MediaObjects/11042_2011_759_Fig14_HTML.gif)
![](http://media.springernature.com/m312/springer-static/image/art%3A10.1007%2Fs11042-011-0759-4/MediaObjects/11042_2011_759_Fig15_HTML.gif)
![](http://media.springernature.com/m312/springer-static/image/art%3A10.1007%2Fs11042-011-0759-4/MediaObjects/11042_2011_759_Fig16_HTML.gif)
Similar content being viewed by others
References
ARToolkit http://www.hitl.washington.edu/artoolkit/. Accessed 26 October 2010
Ballagas R, Szybalski A, Fox A (2004) Patch panel: enabling control-flow interoperability in ubicomp environments. In: Proceedings of the 2nd IEEE International Conference on Pervasive Computing and Communications (PerCom’04), pp 241–252
Beigl M, Gellersen HW, Schmidt A (2001) MediaCups: experience with design and use of computer-augmented everyday objects. Comput Netw 35(4):401–409
Cho Y, Shin K, Choi J, Choi J (2007) Toward dynamic adoption for a user’s situation information in a context-aware workflow system. In: Proceedings of International Conference on Computational Science (ICCS2007), pp 236–243
Dey AK, Abowd GD, Salber D (2001) A conceptual framework and a toolkit for supporting the rapid prototyping of context-aware applications. Hum Comput Interact 16(2–4):97–166
Fujinami K, Nakajima T (2005) Sentient artefact: acquiring user’s context through daily objects. In: Proceedings of the 2nd International Symposium on Ubiquitous Intelligence and Smart Worlds (UISW2005), pp 335–344
Harter A, Hopper A, Steggles P, Ward A, Webster P (1999) The anatomy of a context-aware application. Mobile Computing and Network, pp 59–68
Hashimoto A, Mori N, Funatomi T, Yamakata Y, Kakusho K, Minoh M (2008) Smart kitchen: a user centric cooking support system. In: Proceedings of the 12th International Conference on Information Processing and Management of Uncertainty in Knowledge-Based Systems (IPMU’08), pp 848–854
Herrmann K, Rothermel K, Kortuem G, Dulay N (2008) Adaptable pervasive flows—an emerging technology for pervasive adaptation. In: Proceedings of the 2nd IEEE International Conference on Self-Adaptive and Self-Organizing Systems Workshops (SASOW2008), pp 108–113
Hong JY, Suh EH, Kim SJ (2009) Context-aware systems: a literature review and classification. Expert Syst Appl 36(4):8509–8522
Intille SS, Lee V, Pinhanez C (2003) Ubiquitous computing in the living room: concept sketches and an implementation of a persistent user interface. In: Adjunct Proceedings of UbiComp’03, pp 265–266
Jena Semantic Web Framework. http://jena.sourceforge.net/index.html. Accessed 26 October 2010.
Kaltenbrunner M, Bencina R (2007) reacTIVision: a computer-vision framework for table-based tangible interaction. In: Proceedings of the 1st International Conference on Tangible and Embedded Interaction (TEI’07), pp 69–74
Kawsar F, Fujinami K, Nakajima T (2008) Protoy middleware platform for smart object systems. Int J Smart Home 2(3):1–18
Lassila O, Swick R (1999) Resource Description Framework (RDF) Model and Syntax Specification. http://www.w3.org/TR/1999/REC-rdf-syntax-19990222/. Accessed 26 October 2010
Nintendo (2006) Shaberu! DS Ryouri Navi. http://www.nintendo.com/ds. Accessed 7 January 2011. (A similar title for US customers is “America’s Test Kitchen: Let’s Get Cooking”, http://nintendods.com/atk.jsp.)
OASIS, Web Services Business Process Execution Language Version 2.0 Primer. http://docs.oasis-open.org/wsbpel/2.0/wsbpel-v2.0.pdf. Accessed 10 January 2011.
Raskar R, Baar J, Beardsley P, Willwacher T, Rao S, Forlines C (2003) iLamps: geometrically aware and self-configuring projectors. In: Proceedings of ACM SIGGRAPH, pp 809–818
Roman M, Hess CK, Cerqueira R, Ranganathan A, Campbell RH, Nahrstedt K (2002) Gaia: a middleware infrastructure to enable active spaces. IEEE Pervasive Computing, pp 74–83
Sokan A, Inagawa N, Nishijo K, Shinagawa N, Egi H, Fujinami K (2010) Alerting accidents with ambiguity: a tangible tabletop application for safe and independent chemistry experiments. In: Proceedings of the 7th International Conference on Ubiquitous Intelligence and Computing (UIC2010), pp 151–166
Wieland M, Kopp O, Nicklas D, Leymann F (2007) Towards context-aware workflows. In: Proceedings of the 19th International Conference on Advanced Information Systems Engineering (CAiSE’07)
Wolf H, Herrmann K, Rothermel K (2009) Modeling dynamic context awareness for situated workflows. In: Proceedings of the Confederated International Workshops and Posters on On the Move to Meaningful Internet Systems (OTM’09), pp 98–107
Acknowledgements
We thank Mr. Shogo Hashimoto for developing the AR-Cooking system. This work has been supported by MEXT funds for Grant-in-Aid for Division of Young Researchers and for Scientific Research (C) No. 21500117.
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Fujinami, K., Inagawa, N., Nishijo, K. et al. A middleware for a tabletop procedure-aware information display. Multimed Tools Appl 57, 269–293 (2012). https://doi.org/10.1007/s11042-011-0759-4
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11042-011-0759-4