Abstract
Motion effects are the most important physical effects of commercial feature films in the latest cinemas equipped with multi-sensory-effect systems known as 4DX theaters. This paper presents a framework for automatically generating motion effects from a video. We propose a seven-step motion-effect production pipeline based on the concept of structure from motion. In particular, we present the total-variation-based adaptive noise reduction method to remove acceleration noise and introduce methods to estimate the gravity direction from an image. The proposed framework is validated by prototype application to a synthetic video, a commercial feature film, a ride film, and three point-of-view (POV) footages.
Similar content being viewed by others
References
Heilig, M.L.: Sensorama simulator. US Patent 3,050,870 (1962)
Heilig, M.L.: Experience theater. US Patent 3,469,837 (1969)
Yamashita, A.: Theaters of illusion: the continuing evolution of entertainment simulation. ACM SIGGRAPH Comp. Graph. 28(2), 142–144 (1994)
Hamed, H.M., Hema, R.A.: The application of the 4-D movie theater system in Egyptian museums for enhancing cultural tourism. J. Tour. 10(1), 37–53 (2009)
Hawkins, D.G.: Virtual reality and passive simulators: the future of fun. In: Biocca, F., Levy, M.R. (eds.) Communication in the age of virtual reality. Lawrence Erlbaum Associates, Hillsdale, New Jersey 07642, pp. 159–189 (1995)
CJ 4DX. http://www.cj4dx.com/. Accessed 7 September 2012
CJ CGV. http://www.cgvcinemas.com/. Accessed 7 September 2012
Whitehead, G.: The technological art of simulation. SMPTE Motion Imaging J. 110(1), 39–42 (2001)
Waltl, M.: The next dimension of video experience: sensory effects. IEEE international symposium on a world of wireless, mobile and multimedia networks, Lucca, Italy, pp. 1–3, 20–23 June 2011
Choi, B., Lee, E.S., Yoon, K.: Streaming media with sensory effect. International conference on information science and applications, Jeju, Korea, pp. 1–6 (2011)
Yoo, B., Cha, M., Han, S.: A framework for a multi-sensory VR effect system with motional display. International conference on cyberworlds, Singapore, pp. 237–244 (2005)
Lee, I., Han, S.: Estimating the 3D posture of 6DOF platform from image sequences. International symposium on advanced intelligent systems, Sokcho, Korea, pp. 894–897 (2007)
Parent, R.: Computer animation: algorithms and techniques, 2nd edn. Morgan Kaufmann Publishers, San Francisco (2007)
Hartley, R., Zisserman, A.: Multiple view geometry in computer vision, 2nd edn. Cambridge University Press, New York (2003)
Kemeny, A., Panerai, F.: Evaluating perception in driving simulation experiments. Trends Cognit. Sci. 7(1), 31–37 (2003). doi:10.1016/s1364-6613(02)00011-6
Nahon, M., Reid, L.: Simulator motion-drive algorithms: a designer’s perspective. J. Guid. Control Dyn. 13(2), 356–362 (1990)
Pollefeys, M., Van Gool, L., Vergauwen, M., Verbiest, F., Cornelis, K., Tops, J., Koch, R.: Visual modeling with a hand-held camera. Int. J. Comput. Vis. 59(3), 207–232 (2004)
Snavely, N., Seitz, S.M., Szeliski, R.: Photo tourism: exploring photo collections in 3D. ACM Trans. Graph. 25(3), 835–846 (2006)
Thormahlen, T., Hasler, N., Wand, M., Seidel, HP.: Merging of feature tracks for camera motion estimation from video. European conference on visual media production, London, UK, pp. 1–8, 26–27 November 2008
Nistér, D., Naroditsky, O., Bergen, J.: Visual odometry for ground vehicle applications. J. Field Robotics 23(1), 3–20 (2006)
Davison, A.J., Reid, I.D., Molton, N.D., Stasse, O.: MonoSLAM: real-time single camera SLAM. IEEE Trans. Anal. Mach. Intell. 29(6), 1052–1067 (2007)
Klein, G., Murray, D.: Parallel tracking and mapping for small AR workspaces. International symposium on mixed and augmented reality, Nara, Japan, pp. 225–234, 13–16 November 2007
Mouragnon, E., Lhuillier, M., Dhome, M., Dekeyser, F., Sayd, P.: Generic and real-time structure from motion using local bundle adjustment. Image Vis. Comput. 27(8), 1178–1193 (2009)
Triggs, B., McLauchlan, P., Hartley, R., Fitzgibbon, A.: Bundle adjustment—a modern synthesis. In: Triggs, B., Zisserman, A., Szeliski, R. (eds.) Vision algorithms: theory and practice, vol 1883. Lecture notes in computer science. Springer, Berlin, pp. 298–372. doi:10.1007/3-540-44480-7_21
Boujou. http://www.vicon.com/boujou/. Accessed 7 September 2012
SynthEyes Camera Tracker. http://www.ssontech.com/synovu.htm. Accessed 7 September 2012
MatchMover. http://download.autodesk.com/us/maya/2013help-matchmover/. Accessed 7 September 2012
Voodoo Camera Tracker. http://www.digilab.uni-hannover.de/docs/manual.html. Accessed 7 September 2012
Ladin, Z., Flowers, W.C., Messner, W.: A quantitative comparison of a position measurement system and accelerometry. J. Biomech. 22(4), 295–308 (1989)
Tilli, A., Montanari, M.: A low-noise estimator of angular speed and acceleration from shaft encoder measurement. AUTOMATIKA 42(3–4), 169–176 (2001)
Bélanger, P.R., Dobrovolny, P., Helmy, A., Zhang, X.: Estimation of angular velocity and acceleration from shaft-encoder measurements. Int. J. Robotics Res. 17(11), 1225–1233 (1998). doi:10.1177/027836499801701107
Zhou, J., Shen, X., Petriu, EM., Georganas, N.D.: Linear velocity and acceleration estimation of 3 DOF haptic interfaces. IEEE international workshop on haptic audio visual environments and games, Ottawa, ON, Canada, pp. 137–142, 18–19 October 2008
Brown, R.H., Schneider, S.C., Mulligan, M.G.: Analysis of algorithms for velocity estimation from discrete position versus time data. IEEE Trans. Ind. Electron. 39(1), 11–19 (1992)
Leifer, J., Weems, B., Kienle, S., Sims, A.: Three-dimensional acceleration measurement using videogrammetry tracking data. Exp. Mech. 51(2), 199–217 (2011)
Savitzky, A., Golay, M.J.E.: Smoothing and differentiation of data by simplified least squares procedures. Anal. Chem. 36(8), 1627–1639 (1964). doi:10.1021/ac60214a047
Bos, J.E., MacKinnon, S.N., Patterson, A.: Motion sickness symptoms in a ship motion simulator: effects of inside, outside, and no view. Aviat. Space Environ. Med 76(12), 1111–1118 (2005)
Rudin, L.I., Osher, S., Fatemi, E.: Nonlinear total variation based noise removal algorithms. Phys D Nonlinear Phenom. 60(1–4), 259–268 (1992). doi:10.1016/0167-2789(92)90242-f
Ballard, D.H.: Generalizing the Hough transform to detect arbitrary shapes. Pattern Recognit. 13(2), 111–122 (1981)
Cinemetrics. http://www.cinemetrics.lv/. Accessed 5 February 2013
California Screamin’ Roller Coaster Front Seat POV. http://youtu.be/08VwtYdoBhM?t=1m30s. Accessed 5 February 2013
Downhill mountain biking helmet cam POV Winter Park Colorado. http://youtu.be/HAzWivQ6v7I?t=2m50s. Accessed 5 February 2013
GoPro: Horse Show Jumping. http://youtu.be/7ZzWccM_5cM?t=10s. Accessed 5 February 2013
Night Landing in Los Angeles (cockpit view). http://youtu.be/OpA3ORYlgGs?t=50s. Accessed 5 February 2013
Oh, H.: 4D—The future of cinema (in Korean). The Chosunilbo. http://news.chosun.com/site/data/html_dir/2011/06/29/2011062902540.html. Accessed 5 February 2013
Acknowledgment
This work was supported in part by Agency for Defense Development under the contract UD110039DD, Korea and the Korea Institute of Science and Technology (KIST) Institutional Program (Project No. 2E24100).
Author information
Authors and Affiliations
Corresponding author
Additional information
Communicated by B. Prabhakaran.
Rights and permissions
About this article
Cite this article
Shin, S., Yoo, B. & Han, S. A framework for automatic creation of motion effects from theatrical motion pictures. Multimedia Systems 20, 327–346 (2014). https://doi.org/10.1007/s00530-013-0322-4
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s00530-013-0322-4