Skip to main content
SpringerLink
Log in

Personalized multimedia content delivery on an interactive table by passive observation of museum visitors

  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

The amount of multimedia data collected in museum databases is growing fast, while the capacity of museums to display information to visitors is acutely limited by physical space. Museums must seek the perfect balance of information given on individual pieces in order to provide sufficient information to aid visitor understanding while maintaining sparse usage of the walls and guaranteeing high appreciation of the exhibit. Moreover, museums often target the interests of average visitors instead of the entire spectrum of different interests each individual visitor might have. Finally, visiting a museum should not be an experience contained in the physical space of the museum but a door opened onto a broader context of related artworks, authors, artistic trends, etc. In this paper we describe the MNEMOSYNE system that attempts to address these issues through a new multimedia museum experience. Based on passive observation, the system builds a profile of the artworks of interest for each visitor. These profiles of interest are then used to drive an interactive table that personalizes multimedia content delivery. The natural user interface on the interactive table uses the visitor’s profile, an ontology of museum content and a recommendation system to personalize exploration of multimedia content. At the end of their visit, the visitor can take home a personalized summary of their visit on a custom mobile application. In this article we describe in detail each component of our approach as well as the first field trials of our prototype system built and deployed at our permanent exhibition space at LeMurate (http://www.lemurate.comune.fi.it/lemurate/) in Florence together with the first results of the evaluation process during the official installation in the National Museum of Bargello (http://www.uffizi.firenze.it/musei/?m=bargello).

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11

Similar content being viewed by others

Notes

  1. Mnemosyne (the source of the word mnemonic) was a titan and the personification of memory in Greek mythology. We chose the name MNEMOSYNE for the project to emphasize the assistive nature of the proposed system in acquiring new information.

  2. http://mahout.apache.org

  3. http://www.cvg.rdg.ac.uk/PETS2009/index.html

References

  1. Anderson P, Blackwood A (2004) Mobile and pda technologies and their future use in education. In: Proceedings of JISC Technology and Standards Watch, vol. 4, pp 3–33

  2. Avraham T, Gurvich I, Lindenbaum M, Markovitch S (2012) Learning implicit transfer for person re-identification. In: Proceedings of ECCV - Workshops and Demonstrations, pp 381–390

  3. Baber C, Bristow H, Cheng SL, Hedley A, Kuriyama Y, Lien M, Pollard J, Sorrell P (2001) Augmenting museums and art galleries. In: 13rd Conference on Human-Computer Interaction Interact 2001, pp 439–447

  4. Bagdanov AD, Del Bimbo A, Di Fina D, Karaman S, Lisanti G, Masi I (2013) Multi-target data association using sparse reconstruction. In: Proceedings of International Conference on Image Analysis and Processing (ICIAP), pp 239–248

  5. Bagdanov AD, Del Bimbo A, Landucci L, Pernici F (2011) Mnemosyne: Enhancing the museum experience through interactive media and visual profiling, In: Multimedia for Cultural Heritage, pp 39–50. Springer

  6. Ballagas R, Rohs M, Sheridan JG, Borchers J (2004) Byod: Bring your own device. In: UbiComp 2004 Workshop on Ubiquitous Display Environments

  7. Bangor A, Kortum PT, Miller JT (2008) An empirical evaluation of the system usability scale. Intl J Human– Comput Interact 24(6):574–594

    Article  Google Scholar 

  8. Bartoli F, Lisanti G, Karaman S, Bagdanov AD, Bimbo AD (2014) Unsupervised scene adaptation for faster multi-scale pedestrian detection. In: Proceedings of the 22nd International Conference on Pattern Recognition (ICPR 2014)

  9. Bay H, Fasel B, Van Gool L (2006) Interactive museum guide: Fast and robust recognition of museum objects. In: Proceedings of the first international workshop on mobile vision

  10. Bazzani L, Cristani M, Perina A, Murino V (2012) Multiple-shot person re-identification by chromatic and epitomic analyses. Pattern Recogn Lett 33(7):898–903

    Article  Google Scholar 

  11. Billsus D, Pazzani MJ (2000) User modeling for adaptive news access. User Model User-Adap Inter 10(2-3):147–180

    Article  Google Scholar 

  12. Bimbo AD, Lisanti G, Masi I, Pernici F (2010) Person detection using temporal and geometric context with a pan tilt zoom camera. In: 20th international conference on pattern recognition (ICPR). IEEE, pp 3886–3889

  13. Bowen JP (2010) A brief history of early museums online. The Rutherford Journal

  14. Bowen JP, Filippini-Fantoni S (2004) Personalization and the web from a museum perspective, In: Proceedings of the museums and the web conference (MW2004)

  15. Breitenstein MD, Reichlin F, Leibe B, Koller-Meier E, Van Gool L (2009) Robust tracking-by-detection using a detector confidence particle filter, In: 12th international conference on computer vision, pp 1515–1522

  16. Breitenstein MD, Reichlin F, Leibe B, Koller-Meier E, Van Gool L (2011) Online multiperson tracking-by-detection from a single, uncalibrated camera. IEEE Trans Patt Anal Machine Intell 33(9):1820–1833

    Article  Google Scholar 

  17. Bruns E, Brombach B, Zeidler T, Bimber O (2007) Enabling mobile phones to support large-scale museum guidance. MultiMe IEEE 14(2):16–25

    Article  Google Scholar 

  18. Brusilovsky P, Maybury MT (2002) From adaptive hypermedia to the adaptive web. Commun ACM 45(5):30–33

    Article  Google Scholar 

  19. Cheng DS, Cristani M, Stoppa M, Bazzani L, Murino V (2011) Custom pictorial structures for re-identification. In: Proceedings of BMVC, vol. 2, p 6

  20. Dalal N, Triggs B (2005) Histograms of oriented gradients for human detection. In: IEEE computer society conference on computer vision and pattern recognition, 2005. CVPR 2005, vol. 1, pp 886– 893

  21. Delgado J, Ishii N (1999) On-line learning of user preferences in recommender systems. In: Proceedings of international joint conference on artificial intelligence (IJCAI-99), Workshop on machine learning for information filtering, Stockholm, Sweden

  22. Farenzena M, Bazzani L, Perina A, Murino V, Cristani M (2010) Person re-identification by symmetry-driven accumulation of local features. In: Proceedings of CVPR: 2360–2367

  23. Felzenszwalb PF, Girshick RB, McAllester D, Ramanan D (2010) Object detection with discriminatively trained part-based models. IEEE Trans Patt Anal Mach Intell 32(9):1627–1645

    Article  Google Scholar 

  24. Hartley RI, Zisserman A (2004) Multiple view geometry in computer vision, 2nd edn. Cambridge University Press, ISBN: 0521540518

  25. Hatala M, Wakkary R (2005) User modeling and semantic technologies in support of a tangible interface. JUser Model User Adap Interact 15(3-4):339–380

    Article  Google Scholar 

  26. Huang Z, Chen H, Zeng D (2004) Applying associative retrieval techniques to alleviate the sparsity problem in collaborative filtering. ACM Trans Inf Syst (TOIS) 22(1):116–142

    Article  Google Scholar 

  27. Hyvönen E (2009) Semantic portals for cultural heritage, In: Handbook on ontologies, pp 757–778. Springer

  28. Jin C, Takahashi S, Tanaka J (2006) Interaction between small size device and large screen in public space. In: Knowledge-based intelligent information and engineering systems, pp 197–204, Springer

  29. Karaman S, Bagdanov AD (2012) Identity inference: generalizing person re-identification scenarios. In: Computer Vision–ECCV 2012. Workshops and demonstrations, pp 443–452, Springer

  30. Karaman S, Bagdanov AD, D’Amico G, Landucci L, Ferracani A, Pezzatini D, Del Bimbo A (2013) Passive profiling and natural interaction metaphors for personalized multimedia museum experiences, In: New trends in image analysis and processing–ICIAP 2013, pp 247–256. Springer Berlin Heidelberg

  31. Karaman S, Lisanti G, Bagdanov AD, Bimbo AD Leveraging local neighborhood topology for large scale person re-identification. Pattern Recognition (2014). doi:10.1016/j.patcog.2014.06.003. In press

  32. Karaman S, Lisanti G, Bagdanov AD, Del Bimbo A (2013) From re-identification to identity inference: labelling consistency by local similarity constraints, In: Person re-identification, advances in computer vision and pattern recognition, pp 287–307, Springer

  33. Kuflik T, Stock O, Zancanaro M, Gorfinkel A, Jbara S, Kats S, Sheidin J, Kashtan N (2011) A visitor’s guide in an active museum: Presentations, communications, and reflection. J Comput Cultural Herit (JOCCH) 3(3):11

    Google Scholar 

  34. Kumar R, Raghavan P, Rajagopalan S, Tomkins A (1998) Recommendation systems: a probabilistic analysis. In: Proceedings of the 39th annual symposium on foundations of computer science 1998, pp 664–673

  35. Leibe B, Schindler K, Cornelis N, Van Gool L (2008) Coupled object detection and tracking from static cameras and moving vehicles. Patt Anal Mach Intell IEEE Trans 30(10):1683–1698

    Article  Google Scholar 

  36. Leibe B, Seemann E, Schiele B (2005) Pedestrian detection in crowded scenes. In: IEEE computer society conference on computer vision and pattern recognition, 2005. CVPR 2005, vol. 1, pp 878–885

  37. Mikolajczyk K, Schmid C, Zisserman A (2004) Human detection based on a probabilistic assembly of robust part detectors. In: Computer Vision-ECCV 2004, pp 69–82

  38. Negroponte N (1996) Being digital. Random House LLC

  39. Norman DA (1999) The invisible computer: why good products can fail, the personal computer is so complex, and information appliances are the solution. MIT press

  40. Papageorgiou C, Poggio T (1999) Trainable pedestrian detection. In: Proceedings of the 1999 international conference on image processing, 1999. ICIP 99, vol. 4, pp 35–39

  41. Pechenizkiy M, Calders T (2007) A framework for guiding the museum tours personalization. In: Proceedings of the workshop on personalized access to Cultural Heritage (PATCH07), pp 11– 28

  42. Popescul A, Pennock DM, Lawrence S (2001) Probabilistic models for unified collaborative and content-based recommendation in sparse-data environments. In: Proceedings of the seventeenth conference on uncertainty in artificial intelligence, pp 437–444

  43. Prosser B, Zheng WS, Gong S, Xiang T (2010) Person re-identification by support vector ranking. In: Proceedings of BMVC

  44. Rashid AM, Albert I, Cosley D, Lam SK, McNee SM, Konstan JA, Riedl J (2002) Getting to know you: learning new user preferences in recommender systems. In: Proceedings of the 7th international conference on Intelligent user interfaces, pp 127–134

  45. Rukzio E, Schmidt A, Hussmann H (2004) An analysis of the usage of mobile phones for personalized interactions with ubiquitous public displays. In: Workshop ubiquitous display environments in conjunction with UbiComp

  46. Sakamura K (2003) Personalized digital museum assistant. Digital Museum: 2000

  47. Sampaio I, Ramalho G, Corruble V, Prudêncio R (2006) Acquiring the preferences of new users in recommender systems-the role of item controversy. In: Proceedings of the ECAI 2006 workshop on recommender systems, pp 107–110

  48. Savia E, Puolamäki K, Sinkkonen J, Kaski S (2005) Two-way latent grouping model for user preference prediction. In: Proceedings of the UAI’05, Citeseer

  49. Sparacino F (2002) The museum wearable: real-time sensor-driven understanding of visitors’ interests for personalized visually-augmented museum experiences, In: Proceedings of Museums and the Web (MW2002)

  50. Sparacino F (2004) Scenographies of the past and museums of the future: from the wunderkammer to body-driven interactive narrative spaces. In: Proceedings of the 12th annual ACM international conference on multimedia, pp 72–79. ACM

  51. Starner T, Mann S, Rhodes B, Levine J, Healey J, Kirsch D, Picard RW, Pentland A (1997) Augmented reality through wearable computing. Presence: Teleoperators Virtual Environ 6(4):386–398

    Article  Google Scholar 

  52. Stock O, Not E, Zancanaro M (2005) Intelligent interactive information presentation for cultural tourism. In: Multimodal intelligent information presentation, pp 95–111, Springer

  53. Ungar LH, Foster DP (1998) Clustering methods for collaborative filtering. In: AAAI Workshop on recommendation systems, vol 1

  54. Wang X, Han TX, Yan S (2009) An hog-lbp human detector with partial occlusion handling. In: IEEE 12th International Conference on Computer Vision, pp 32–39

  55. Wang Y, Van Sambeek R, Schuurmans Y, Aroyo L, Stash N, Rutledge L, Gorgels P (2008) Be your own curator with the chip tour wizard. In: Proceedings of the museums and the web conference (MW2008)

  56. Wang Y, Stash N, Sambeek R, Schuurmans Y, Aroyo L, Schreiber G, Gorgels P (2009) Cultivating personalized museum tours online and on-site. Interdisc Sci Rev 34(2-3):139–153

    Article  Google Scholar 

  57. Artabus http://www.artabus.com/. Accessed: 2014-03-18

  58. Art.net http://www.art.net/. Accessed: 2014-03-18

  59. Peach home page http://peach.fbk.eu/. Accessed: 2014-04-07

  60. Polo museale fiorentino :: Sito ufficiale http://www.uffizi.firenze.it. Accessed: 2014-03-18

  61. The san francisco museum of modern art (sfmoma) http://www.sfmoma.org/. Accessed: 2014-03-18

  62. Site officiel du musée du louvre http://www.louvre.fr. Accessed: 2014-03-18

  63. Tate http://www.tate.org.uk/. Accessed: 2014-03-18

  64. Virtual museum exhibit home page http://www.virtualmuseumexhibit.com/. Accessed: 2014-03-18

  65. Web gallery of art http://www.wga.hu/. Accessed: 2014-03-18

  66. Wu B, Nevatia R (2005) Detection of multiple, partially occluded humans in a single image by bayesian combination of edgelet part detectors, In: 10th IEEE international conference on computer vision, vol 1, pp 90–97

  67. Zancanaro M, Stock O, Alfaro I (2003) Using cinematic techniques in a multimedia museum guide, In: Proceedings of museums and the web 2003

Download references

Acknowledgments

This work was partially supported by Thales Italia and the MNEMOSYNE project (POR-FSE 2007-2013, A.IV-OB.2). Andrew D. Bagdanov acknowledges the support of Ramon y Cajal Fellowship RYC-2012-11776.

Author information

Authors and Affiliations

  1. Media Integration and Communication Center (MICC), University of Florence, Firenze, Italy

    Svebor Karaman, Lea Landucci, Gianpaolo D’Amico, Andrea Ferracani, Daniele Pezzatini & Alberto Del Bimbo

  2. Computer Vision Center, Barcelona, Spain

    Andrew D. Bagdanov

Authors
  1. Svebor Karaman
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Andrew D. Bagdanov
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Lea Landucci
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Gianpaolo D’Amico
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Andrea Ferracani
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Daniele Pezzatini
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Alberto Del Bimbo
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Svebor Karaman.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Karaman, S., Bagdanov, A.D., Landucci, L. et al. Personalized multimedia content delivery on an interactive table by passive observation of museum visitors. Multimed Tools Appl 75, 3787–3811 (2016). https://doi.org/10.1007/s11042-014-2192-y

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-014-2192-y

Keywords

Search

Navigation