Tommy Hasselman
Stefanie Zollmann
ABSTRACT
Augmented Reality (AR) opens up new possibilities for interactive experiences which can be used in a variety of circumstances. Rephotography is a photo technique commonly presented on dedicated internet pages that align a past view with a current photo, allowing you to have a comparative view of the past with the present. This project aims to combine these two concepts to create AR experiences where you can view buildings and street views from historical photography seamlessly embedded in the present environment. We report on our automated pipeline that can take a historical photograph of a building and produces a textured 3D model that can be placed in AR over the current view of the building using techniques from machine learning while also reporting on first feedback from a preliminary user study.
Footnotes
Supplemental Material
- Soonmin Bae, Aseem Agarwala, and Fredo Durand. 2010. Computational rephotography. ACM Transactions on Graphics 29, 3 (July 2010), 24:1–24:15. https://doi.org/10.1145/1805964.1805968Google Scholar
Digital Library
- George Caridakis and John Aliprantis. 2019. A Survey of Augmented Reality Applications in Cultural Heritage. Int. J. Comput. Methods Herit. Sci. 3, 2 (jul 2019), 118 – 147. https://doi.org/10.4018/IJCMHS.2019070107Google ScholarDigital Library
- Marco Cavallo, Geoffrey Alan Rhodes, and Angus Graeme Forbes. 2016. Riverwalk: Incorporating Historical Photographs in Public Outdoor Augmented Reality Experiences. In 2016 IEEE International Symposium on Mixed and Augmented Reality (ISMAR-Adjunct). 160–165. https://doi.org/10.1109/ISMAR-Adjunct.2016.0068Google ScholarCross Ref
- A. Criminisi, I. Reid, and A. Zisserman. 2000. Single View Metrology. International Journal of Computer Vision 40, 2 (Nov. 2000), 123–148. https://doi.org/10.1023/A:1026598000963Google ScholarDigital Library
- Paul E. Debevec, Camillo J. Taylor, and Jitendra Malik. 1996. Modeling and rendering architecture from photographs: a hybrid geometry- and image-based approach. In Proceedings of the 23rd annual conference on Computer graphics and interactive techniques(SIGGRAPH ’96). Association for Computing Machinery, New York, NY, USA, 11–20. https://doi.org/10.1145/237170.237191Google ScholarDigital Library
- Richard Hartley and Andrew Zisserman. 2004. Multiple View Geometry in Computer Vision (2 ed.). Cambridge University Press, Cambridge. https://doi.org/10.1017/CBO9780511811685Google ScholarCross Ref
- Derek Hoiem, Alexei A. Efros, and Martial Hebert. 2005. Automatic Photo Pop-Up. In ACM SIGGRAPH 2005 Papers (Los Angeles, California) (SIGGRAPH ’05). Association for Computing Machinery, New York, NY, USA, 577 – 584. https://doi.org/10.1145/1186822.1073232Google ScholarDigital Library
- BKP Horn. 1987. Closed-form solution of absolute orientation using unit quaternions. JOSA A 4, April (1987), 629–642. http://www.opticsinfobase.org/abstract.cfm?&id=2711http://www.opticsinfobase.org/abstract.cfm?&id=2711Google ScholarCross Ref
- Youichi Horry, Ken Anjyo, and Kiyoshi Arai. 1997. Tour Into the Picture: Using Spidery Mesh Interface to Make Animation from a Single Image”. 225–232. https://doi.org/10.1145/258734.258854Google ScholarDigital Library
- Ibrahim Ilhan and Evrim Celtek. 2016. Mobile Marketing: Usage of Augmented Reality in Tourism. Gaziantep University Journal of Social Sciences 15, 2 (Dec. 2016), 581–599. https://doi.org/10.21547/jss.256721Google ScholarCross Ref
- Gun A. Lee, Andreas Dünser, Seungwon Kim, and Mark Billinghurst. 2012. CityViewAR: A mobile outdoor AR application for city visualization. In 2012 IEEE International Symposium on Mixed and Augmented Reality - Arts, Media, and Humanities (ISMAR-AMH). 57–64. https://doi.org/10.1109/ISMAR-AMH.2012.6483989 ISSN: 2381-8360.Google ScholarCross Ref
- Bruce MacEvoy. 2015. Two Point Perspective. In Handprint Watercolors. https://www.handprint.com/HP/WCL/perspect3.htmlGoogle Scholar
- Andrii Matviienko, Sebastian Günther, Sebastian Ritzenhofen, and Max Mühlhäuser. 2022. AR Sightseeing: Comparing Information Placements at Outdoor Historical Heritage Sites Using Augmented Reality. Proc. ACM Hum.-Comput. Interact. 6, MHCI, Article 194 (sep 2022), 17 pages. https://doi.org/10.1145/3546729Google ScholarDigital Library
- Thomas Müller, Alex Evans, Christoph Schied, and Alexander Keller. 2022. Instant Neural Graphics Primitives with a Multiresolution Hash Encoding. ACM Trans. Graph. 41, 4, Article 102 (July 2022), 15 pages. https://doi.org/10.1145/3528223.3530127Google ScholarDigital Library
- Gen Nishida, Adrien Bousseau, and Daniel Aliaga. 2018. Procedural Modeling of a Building from a Single Image. Computer Graphics Forum 37 (05 2018), 415–429. https://doi.org/10.1111/cgf.13372Google ScholarCross Ref
- Kari Rainio, Petri Honkamaa, and Kaisa Spilling. 2015. Presenting Historical Photos using Augmented Reality. Technical Report. VTT Technical Research Centre of Finland.Google Scholar
- S. Rusinkiewicz and M. Levoy. 2001. Efficient variants of the ICP algorithm. In Proceedings Third International Conference on 3-D Digital Imaging and Modeling. 145–152. https://doi.org/10.1109/IM.2001.924423Google ScholarCross Ref
- Ashutosh Saxena, Min Sun, and Andrew Y. Ng. 2009. Make3D: Learning 3D Scene Structure from a Single Still Image. IEEE Transactions on Pattern Analysis and Machine Intelligence 31, 5 (May 2009), 824–840. https://doi.org/10.1109/TPAMI.2008.132Google ScholarDigital Library
- Roman Suvorov, Elizaveta Logacheva, Anton Mashikhin, Anastasia Remizova, Arsenii Ashukha, Aleksei Silvestrov, Naejin Kong, Harshith Goka, Kiwoong Park, and Victor Lempitsky. 2021. Resolution-robust Large Mask Inpainting with Fourier Convolutions. arXiv preprint arXiv:2109.07161 (2021).Google Scholar
- Daniel Wagner, Gerhard Reitmayr, Alessandro Mulloni, Tom Drummond, and Dieter Schmalstieg. 2008. Pose tracking from natural features on mobile phones. In 2008 7th IEEE/ACM International Symposium on Mixed and Augmented Reality. 125–134. https://doi.org/10.1109/ISMAR.2008.4637338Google ScholarDigital Library
- Yuxin Wu, Alexander Kirillov, Francisco Massa, Wan-Yen Lo, and Ross Girshick. 2019. Detectron2. https://github.com/facebookresearch/detectron2.Google Scholar
- Alex Yu, Vickie Ye, Matthew Tancik, and Angjoo Kanazawa. 2021. pixelNeRF: Neural Radiance Fields From One or Few Images. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). 4578–4587.Google ScholarCross Ref
Index Terms
- ARephotography: Revisiting Historical Photographs using Augmented Reality
Recommendations
Haptics in Augmented Reality
ICMCS '99: Proceedings of the IEEE International Conference on Multimedia Computing and Systems - Volume 2An augmented reality system merges synthetic sensory information into a user's perception of a three-dimensional environment. An important performance goal for an augmented reality system is that the user perceives a single seamless environment. In most ...
Extending Virtual Reality Display Wall Environments Using Augmented Reality
SUI '19: Symposium on Spatial User InteractionTwo major form factors for virtual reality are head-mounted displays and large display environments such as CAVE®and the LCD-based successor CAVE2®. Each of these has distinct advantages and limitations based on how they’re used. This work explores ...
Augmented reality as perceptual reality
VSMM'06: Proceedings of the 12th international conference on Interactive Technologies and Sociotechnical SystemsAs shown in Paul Milgram et al’s Reality-Virtuality Continuum (1994), Augmented Reality occupies a very unique status in the spectrum of Mixed Reality. Unlike Virtual Reality, which is completely made up of the virtual and has been the most important ...
Comments