research-article

Open access

Eulerian video magnification and analysis

Authors:

Michael Rubinstein,

Gautham J. Mysore,

Justin G. Chen,

Oral Buyukozturk,

John V. Guttag,

William T. Freeman,

Frédo DurandAuthors Info & Claims

Communications of the ACM, Volume 60, Issue 1

Pages 87 - 95

https://doi.org/10.1145/3015573

Published: 20 December 2016 Publication History

All formats PDF

Abstract

The world is filled with important, but visually subtle signals. A person's pulse, the breathing of an infant, the sag and sway of a bridge---these all create visual patterns, which are too difficult to see with the naked eye. We present Eulerian Video Magnification, a computational technique for visualizing subtle color and motion variations in ordinary videos by making the variations larger. It is a microscope for small changes that are hard or impossible for us to see by ourselves. In addition, these small changes can be quantitatively analyzed and used to recover sounds from vibrations in distant objects, characterize material properties, and remotely measure a person's pulse.

References

[1]

Amir-Khalili, A., Peyrat, J.-M., Abinahed, J., Al-Alao, O., Al-Ansari, A., Hamarneh, G., Abugharbieh, R. Auto localization and segmentation of occluded vessels in robot-assisted partial nephrectomy. In MICCAI 2014 (2014). Springer, 407--414.

[2]

Chen, J.G., Wadhwa, N., Cha, Y.-J., Durand, F., Freeman, W.T., Buyukozturk, O. Structural modal identification through high speed camera video: Motion magnification. In Topics in Modal Analysis I. Volume 7 (2014). Springer, 191--197.

[3]

Chen, J.G., Wadhwa, N., Cha, Y.-J., Durand, F., Freeman, W.T., Buyukozturk, O. Modal identification of simple structures with high-speed video using motion magnification. Journal of Sound and Vibration 345 (2015), 58--71.

[4]

Chen, J.G., Wadhwa, N., Durand, F., Freeman, W.T., Buyukozturk, O. Developments with motion magnification for structural modal identification through camera video. In Dynamics of Civil Structures. Volume 2 (2015). Springer, 49--57.

[5]

Davis, A., Bouman, K.L., Chen, J.G., Rubinstein, M., Durand, F., Freeman, W.T. Visual vibrometry: Estimating material properties from small motions in video. In CVPR 2015 (2015), 5335--5343.

[6]

Davis, A., Chen, J.G., Durand, F. Imagespace modal bases for plausible manipulation of objects in video. ACM Trans. Graph. 34, 6 (2015), 239.

Digital Library

[7]

Davis, A., Rubinstein, M., Wadhwa, N., Mysore, G.J., Durand, F., Freeman, W.T. The visual microphone: Passive recovery of sound from video. ACM Trans. Graph. 33, 4 (2014), 79.

Digital Library

[8]

Elgharib, M.A., Hefeeda, M., Durand, F., Freeman, W.T. Video magnification in presence of large motions. In CVPR 2015 (2015), 4119--4127.

[9]

Fleet, D.J., Jepson, A.D. Computation of component image velocity from local phase information. Int. J. Comput. Vision 5, 1 (1990), 77--104.

Digital Library

[10]

Fuchs, M., Chen, T., Wang, O., Raskar, R., Seidel, H.-P., Lensch, H.P. Real-time temporal shaping of high-speed video streams. Comput. Graph. 34, 5 (2010), 575--584.

Digital Library

[11]

Gautama, T., Van Hulle, M.M. A phase-based approach to the estimation of the optical flow field using spatial filtering. IEEE Trans. Neural Netw. 13, 5 (2002), 1127--1136.

Digital Library

[12]

Horn, B.K., Schunck, B.G. Determining optical flow. In 1981 Technical Symposium East (1981). International Society for Optics and Photonics, 319--331.

Digital Library

[13]

Liu, C., Torralba, A., Freeman, W.T., Durand, F., Adelson, E.H. Motion magnification. ACM Trans. Graph. 24, 3 (2005), 519--526.

Digital Library

[14]

Lucas, B.D., Kanade, T., et al. An iterative image registration technique with an application to stereo vision. In IJCAI. Volume 81 (1981), 674--679.

Digital Library

[15]

McLeod, A.J., Baxter, J.S., de Ribaupierre, S., Peters, T.M. Motion magnification for endoscopic surgery. In SPIE Medical Imaging (2014), 90360C--90360C.

[16]

Poh, M.-Z., McDuff, D.J., Picard, R.W. Non-contact, automated cardiac pulse measurements using video imaging and blind source separation. Opt. Express 18, 10 (2010), 10762--10774.

[17]

Rubinstein, M. Analysis and visualization of temporal variations in video. PhD thesis, Massachusetts Institute of Technology, Cambridge, MA (2014).

[18]

Sellon, J.B., Farrahi, S., Ghaffari, R., Freeman, D.M. Longitudinal spread of mechanical excitation through tectorial membrane traveling waves. Proc. Natl. Acad. Sci. 112, 42 (2015), 12968--12973.

[19]

Simoncelli, E.P., Freeman, W.T. The steerable pyramid: A flexible architecture for multi-scale derivative computation. In ICIP 1995 (1995), 3444.

Digital Library

[20]

Simoncelli, E.P., Freeman, W.T., Adelson, E.H., Heeger, D.J. Shiftable multi-scale transforms. IEEE Trans. Info. Theory 2, 38 (1992), 587--607.

Digital Library

[21]

Wachel, J., Morton, S.J., Atkins, K.E. Piping vibration analysis. In Proceedings of the 19th Turbomachinery Symposium. (1990).

[22]

Wadhwa, N., Rubinstein, M., Durand, F., Freeman, W.T. Phase-based video motion processing. ACM Trans. Graph. 32, 4 (2013), 80.

Digital Library

[23]

Wadhwa, N., Rubinstein, M., Durand, F., Freeman, W.T. Riesz pyramids for fast phase-based video magnification. In ICCP2014 (2014). IEEE, 1--10.

[24]

Wu, H.-Y., Rubinstein, M., Shih, E., Guttag, J. V., Durand, F., Freeman, W.T. Eulerian video magnification for revealing subtle changes in the world. ACM Trans. Graph. 31, 4 (2012), 65.

Digital Library

Cited By

Yang XYan PLiu XLu WZhang XChen MWang G(2025)A novel 3D vibration monitoring method using a monocular camera and phase difference Gradient-based algorithmMeasurement10.1016/j.measurement.2025.116909248(116909)Online publication date: May-2025
https://doi.org/10.1016/j.measurement.2025.116909
Lima JMiosso CFarias M(2025)SynFlowMap: A synchronized optical flow remapping for video motion magnificationSignal Processing: Image Communication10.1016/j.image.2024.117203130(117203)Online publication date: Jan-2025
https://doi.org/10.1016/j.image.2024.117203
Zhu AGong XZhou JZhang XZhang D(2024)Efficient Vibration Measurement and Modal Shape Visualization Based on Dynamic Deviations of Structural Edge ProfilesSensors10.3390/s2413441324:13(4413)Online publication date: 8-Jul-2024
https://doi.org/10.3390/s24134413
Show More Cited By

Index Terms

Eulerian video magnification and analysis
1. Computing methodologies
  1. Artificial intelligence
    1. Computer vision
      1. Computer vision problems
        Video segmentation
      2. Computer vision tasks
        Scene understanding

Recommendations

An overview of Eulerian video motion magnification methods
Abstract
The concept of video motion magnification has become increasingly relevant due to its ability to detect small and invisible motions that can be of great value in a variety of applications. A variety of approaches have been developed to magnify ...
Graphical abstract

Display Omitted
Highlights
- Detecting Imperceptible Motions: Explore motion magnification’s role in revealing subtle movements.
- Demystifying Eulerian Processing: Simplify complex concepts, mathematical foundations.
- Real-World Applications: Illustrate ...
Human Motion Analysis in Video Sequences for Telerehabilitation Systems
Video Guichê

Comments

Information & Contributors

Information

Published In

cover image Communications of the ACM

Communications of the ACM Volume 60, Issue 1

January 2017

95 pages

ISSN:0001-0782

EISSN:1557-7317

DOI:10.1145/3028256

Editor:
Moshe Y. Vardi
Association for Computing Machinery, New York, NY

Issue’s Table of Contents

Copyright © 2016 Owner/Author.

This work is licensed under a Creative Commons Attribution-NonCommercial International 4.0 License.

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 20 December 2016

Published in CACM Volume 60, Issue 1

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Qualifiers

Research-article
Research
Refereed

Funding Sources

Microsoft Research
Defense Advanced Research Projects Agency
Quanta Computer
Shell Research
NSF
Cognex

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

80
Total Citations
View Citations
33,862
Total Downloads

Downloads (Last 12 months)1,133
Downloads (Last 6 weeks)117

Reflects downloads up to 03 Mar 2025

Other Metrics

View Author Metrics

Citations

Cited By

Yang XYan PLiu XLu WZhang XChen MWang G(2025)A novel 3D vibration monitoring method using a monocular camera and phase difference Gradient-based algorithmMeasurement10.1016/j.measurement.2025.116909248(116909)Online publication date: May-2025
https://doi.org/10.1016/j.measurement.2025.116909
Lima JMiosso CFarias M(2025)SynFlowMap: A synchronized optical flow remapping for video motion magnificationSignal Processing: Image Communication10.1016/j.image.2024.117203130(117203)Online publication date: Jan-2025
https://doi.org/10.1016/j.image.2024.117203
Zhu AGong XZhou JZhang XZhang D(2024)Efficient Vibration Measurement and Modal Shape Visualization Based on Dynamic Deviations of Structural Edge ProfilesSensors10.3390/s2413441324:13(4413)Online publication date: 8-Jul-2024
https://doi.org/10.3390/s24134413
Ahani SNiknafs NLavoie PHolsti LDumont G(2024)Video-Based Respiratory Rate Estimation for Infants in the NICUIEEE Journal of Translational Engineering in Health and Medicine10.1109/JTEHM.2024.348852312(684-696)Online publication date: 2024
https://doi.org/10.1109/JTEHM.2024.3488523
Ouyang WKilner KXavier RLiu YLu YFeller SPitts KWu MAusra JJones IWu YLuan HTrueb JHigbee-Dempsey EStepien IGhoreishi-Haack NHaney CLi HKozorovitskiy YHeshmati MBanks AGolden SGood CRogers J(2024)An implantable device for wireless monitoring of diverse physio-behavioral characteristics in freely behaving small animals and interacting groupsNeuron10.1016/j.neuron.2024.02.020Online publication date: Mar-2024
https://doi.org/10.1016/j.neuron.2024.02.020
De Santis SSangirardi MAltomare VMeriggi Pde Felice G(2024)Computer vision-based dynamic identification of a reinforced concrete elevated water tankJournal of Civil Structural Health Monitoring10.1007/s13349-024-00817-615:1(105-125)Online publication date: 20-Jul-2024
https://doi.org/10.1007/s13349-024-00817-6
Choong RYap WHum YLai KLing LVodacek ATee Y(2024)Impact of visual enhancement and color conversion algorithms on remote sound recovery from silent videosJournal of the Society for Information Display10.1002/jsid.127532:3(100-113)Online publication date: 12-Mar-2024
https://doi.org/10.1002/jsid.1275
Zhao HZhang XJiang DGu J(2023)Research on Rotating Machinery Fault Diagnosis Based on an Improved Eulerian Video Motion MagnificationSensors10.3390/s2323958223:23(9582)Online publication date: 3-Dec-2023
https://doi.org/10.3390/s23239582
Grech NAgius JSciberras SMicallef NCamilleri KFalzon O(2023)Non-contact Vital Signs Monitoring in Paediatric Anaesthesia – Current Challenges and Future DirectionActa Medica (Hradec Kralove, Czech Republic)10.14712/18059694.2023.1466:2(39-46)Online publication date: 8-Nov-2023
https://doi.org/10.14712/18059694.2023.14
Chen LPeng CZhao B(2023)Novel Multi-Task Learning for Motion MagnificationIEEE Transactions on Circuits and Systems for Video Technology10.1109/TCSVT.2023.326277733:10(5973-5985)Online publication date: 1-Oct-2023
https://dl.acm.org/doi/10.1109/TCSVT.2023.3262777
Show More Cited By

View Options

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Digital Edition

View this article in digital edition.

Digital Edition

Magazine Site

View this article on the magazine site (external)

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Article

Figures

Tables

Media

View Issue’s Table of Contents