skip to main content
10.1145/3641519.3657443acmconferencesArticle/Chapter ViewAbstractPublication PagessiggraphConference Proceedingsconference-collections
research-article
Open access

Temporal acoustic point holography

Published: 13 July 2024 Publication History

Abstract

Holographic acoustic levitation using phased arrays of transducers has facilitated innovative mid-air particle displays. However, current time-invariant methods for computing acoustic holograms often induce high phase changes and thus transducer amplitude fluctuations, severely limiting possible dynamic displays. In this work, we develop a wide range of temporal phase retrieval algorithms that suppress phase change and trade off computational time and solution optimality. We base our hardware implementation on Gerchberg-Saxton (GS), which we identify as gradient descent for maximizing focal amplitudes and thus acoustic trap quality. Following this, we adapt GS to additionally constrain transducers’ (and points’) phase changes from the previous time frame, and enable multiparticle animations without prior knowledge of particles’ paths and interactions. We experimentally showcase a series of levitated character animations depicting human motion and believe our work paves the way for natural dynamic multi-particle displays and unencumbered 3D delivery of other modalities such as haptics and audio.

Supplemental Material

MP4 File - presentation
presentation
ZIP File
Supplementary video, supplementary information and experiment videos
ZIP File
Supplementary video, supplementary information, and experimental videos

References

[1]
Heinz H Bauschke, Patrick L Combettes, and D Russell Luke. 2002. Phase retrieval, error reduction algorithm, and Fienup variants: a view from convex optimization. JOSA A 19, 7 (2002), 1334–1345.
[2]
Amir Beck and Marc Teboulle. 2009. A fast iterative shrinkage-thresholding algorithm for linear inverse problems. SIAM journal on imaging sciences 2, 1 (2009), 183–202.
[3]
Emmanuel J Candes, Xiaodong Li, and Mahdi Soltanolkotabi. 2015. Phase retrieval via Wirtinger flow: Theory and algorithms. IEEE Transactions on Information Theory 61, 4 (2015), 1985–2007.
[4]
Praneeth Chakravarthula, Yifan Peng, Joel Kollin, Henry Fuchs, and Felix Heide. 2019. Wirtinger holography for near-eye displays. ACM Transactions on Graphics (TOG) 38, 6 (2019), 1–13.
[5]
Antonin Chambolle and Thomas Pock. 2011. A first-order primal-dual algorithm for convex problems with applications to imaging. Journal of mathematical imaging and vision 40 (2011), 120–145.
[6]
Jingkai Chen, Jiaoyang Li, Chuchu Fan, and Brian C Williams. 2021. Scalable and safe multi-agent motion planning with nonlinear dynamics and bounded disturbances. In Proceedings of the AAAI Conference on Artificial Intelligence, Vol. 35. 11237–11245.
[7]
Suyeon Choi, Manu Gopakumar, Yifan Peng, Jonghyun Kim, Matthew O’Toole, and Gordon Wetzstein. 2022. Time-multiplexed neural holography: a flexible framework for holographic near-eye displays with fast heavily-quantized spatial light modulators. In ACM SIGGRAPH 2022 Conference Proceedings. 1–9.
[8]
Roberto Di Leonardo, Francesca Ianni, and Giancarlo Ruocco. 2007. Computer generation of optimal holograms for optical trap arrays. Optics Express 15, 4 (2007), 1913–1922.
[9]
Steven Diamond, Vincent Sitzmann, Felix Heide, and Gordon Wetzstein. 2017. Unrolled optimization with deep priors. arXiv preprint arXiv:1705.08041 (2017).
[10]
Veit Elser, Ti-Yen Lan, and Tamir Bendory. 2018. Benchmark problems for phase retrieval. SIAM Journal on Imaging Sciences 11, 4 (2018), 2429–2455.
[11]
Iñigo Ezcurdia, Rafael Morales, Marco AB Andrade, and Asier Marzo. 2022. LeviPrint: Contactless Fabrication using Full Acoustic Trapping of Elongated Parts. In ACM SIGGRAPH 2022 Conference Proceedings. 1–9.
[12]
Daniele Foresti, Majid Nabavi, Mirko Klingauf, Aldo Ferrari, and Dimos Poulikakos. 2013. Acoustophoretic contactless transport and handling of matter in air. Proceedings of the National Academy of Sciences 110, 31 (2013), 12549–12554.
[13]
Tatsuki Fushimi, Asier Marzo, Bruce W Drinkwater, and Thomas L Hill. 2019. Acoustophoretic volumetric displays using a fast-moving levitated particle. Applied Physics Letters 115, 6 (2019).
[14]
Tatsuki Fushimi, Kenta Yamamoto, and Yoichi Ochiai. 2021. Acoustic hologram optimisation using automatic differentiation. Scientific reports 11, 1 (2021), 12678.
[15]
Lei Gao, Pourang Irani, Sriram Subramanian, Gowdham Prabhakar, Diego Martinez Plasencia, and Ryuji Hirayama. 2023. DataLev: Mid-air Data Physicalisation Using Acoustic Levitation. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–14.
[16]
Ralph W Gerchberg. 1972. A practical algorithm for the determination of plane from image and diffraction pictures. Optik 35, 2 (1972), 237–246.
[17]
Saeed Ghorbani, Kimia Mahdaviani, Anne Thaler, Konrad Kording, Douglas James Cook, Gunnar Blohm, and Nikolaus F Troje. 2021. MoVi: A large multi-purpose human motion and video dataset. Plos one 16, 6 (2021), e0253157.
[18]
T Goldstein, E Esser, and R Baraniuk. 2023. Adaptive primal-dual hybrid gradient methods for saddle-point problems. 2013. arXiv preprint arXiv:1305.0546 (2023).
[19]
James Hardwick, Ben Kazemi, Mohamed S Talamali, Giorgos Christopoulos, and Sriram Subramanian. 2023. Actively Reconfigurable Segmented Spatial Sound Modulators. Advanced Materials Technologies (2023), 2301459.
[20]
Felix Heide, Steven Diamond, Matthias Nießner, Jonathan Ragan-Kelley, Wolfgang Heidrich, and Gordon Wetzstein. 2016. Proximal: Efficient image optimization using proximal algorithms. ACM Transactions on Graphics (TOG) 35, 4 (2016), 1–15.
[21]
Ryuji Hirayama, Giorgos Christopoulos, Diego Martinez Plasencia, and Sriram Subramanian. 2022. High-speed acoustic holography with arbitrary scattering objects. Science advances 8, 24 (2022), eabn7614.
[22]
Ryuji Hirayama, Diego Martinez Plasencia, Nobuyuki Masuda, and Sriram Subramanian. 2019. A volumetric display for visual, tactile and audio presentation using acoustic trapping. Nature 575, 7782 (2019), 320–323.
[23]
Takayuki Hoshi. 2016. Gradual phase shift to suppress noise from airborne ultrasound tactile display. In Proc. ACM CHI Workshop Mid-Air Haptics Displays: Syst. Un-instrumented Mid-Air Interact., Session 2: Provide Vis. Haptic Feedback.
[24]
Gunnar Johansson. 1973. Visual perception of biological motion and a model for its analysis. Perception & psychophysics 14 (1973), 201–211.
[25]
Jiatong Li, Chenfeng Gao, and Ken Nakagaki. 2022. ShadowAstro: Levitating Constellation Silhouette for Spatial Exploration and Learning. In Adjunct Proceedings of the 35th Annual ACM Symposium on User Interface Software and Technology. 1–3.
[26]
Benjamin Long, Sue Ann Seah, Tom Carter, and Sriram Subramanian. 2014. Rendering volumetric haptic shapes in mid-air using ultrasound. ACM Transactions on Graphics (TOG) 33, 6 (2014), 1–10.
[27]
James Maas and Gunnar Johansson. 1971. Motion perception, Part I: 2-dimensional motion perception. https://www.youtube.com/watch?v=1F5ICP9SYLU
[28]
Stefano Marchesini. 2007. Phase retrieval and saddle-point optimization. JOSA A 24, 10 (2007), 3289–3296.
[29]
Asier Marzo and Bruce W Drinkwater. 2019. Holographic acoustic tweezers. Proceedings of the National Academy of Sciences 116, 1 (2019), 84–89.
[30]
Asier Marzo, Sue Ann Seah, Bruce W Drinkwater, Deepak Ranjan Sahoo, Benjamin Long, and Sriram Subramanian. 2015. Holographic acoustic elements for manipulation of levitated objects. Nature communications 6, 1 (2015), 8661.
[31]
Kai Melde, Heiner Kremer, Minghui Shi, Senne Seneca, Christoph Frey, Ilia Platzman, Christian Degel, Daniel Schmitt, Bernhard Schölkopf, and Peer Fischer. 2023. Compact holographic sound fields enable rapid one-step assembly of matter in 3D. Science Advances 9, 6 (2023), eadf6182.
[32]
Kai Melde, Andrew G Mark, Tian Qiu, and Peer Fischer. 2016. Holograms for acoustics. Nature 537, 7621 (2016), 518–522.
[33]
Roberto Montano-Murillo, Ryuji Hirayama, and Diego Martinez Plasencia. 2023. OpenMPD: A low-level presentation engine for Multimodal Particle-based Displays. ACM Transactions on Graphics 42, 2 (2023), 1–13.
[34]
Rafael Morales, Asier Marzo, Sriram Subramanian, and Diego Martínez. 2019. LeviProps: Animating levitated optimized fabric structures using holographic acoustic tweezers. In Proceedings of the 32nd Annual ACM Symposium on User Interface Software and Technology. 651–661.
[35]
Masaaki Nagahara. 2015. Discrete signal reconstruction by sum of absolute values. IEEE Signal Processing Letters 22, 10 (2015), 1575–1579.
[36]
Yoichi Ochiai, Takayuki Hoshi, and Jun Rekimoto. 2014. Pixie dust: Graphics generated by levitated and animated objects in computational acoustic-potential field. ACM Transactions on Graphics (TOG) 33, 4 (2014), 1–13.
[37]
Yoichi Ochiai, Takayuki Hoshi, and Ippei Suzuki. 2017. Holographic whisper: Rendering audible sound spots in three-dimensional space by focusing ultrasonic waves. In proceedings of the 2017 CHI conference on human factors in computing systems. 4314–4325.
[38]
Yifan Peng, Suyeon Choi, Nitish Padmanaban, and Gordon Wetzstein. 2020. Neural holography with camera-in-the-loop training. ACM Transactions on Graphics (TOG) 39, 6 (2020), 1–14.
[39]
Diego Martinez Plasencia, Ryuji Hirayama, Roberto Montano-Murillo, and Sriram Subramanian. 2020. GS-PAT: high-speed multi-point sound-fields for phased arrays of transducers. ACM Transactions on Graphics (TOG) 39, 4 (2020), 138–1.
[40]
Jordi Prat-Camps, Giorgos Christopoulos, James Hardwick, and Sriram Subramanian. 2020. A manually reconfigurable reflective spatial sound modulator for ultrasonic waves in air. Advanced Materials Technologies 5, 8 (2020), 2000041.
[41]
Shun Suzuki, Masahiro Fujiwara, Yasutoshi Makino, and Hiroyuki Shinoda. 2020. Reducing amplitude fluctuation by gradual phase shift in midair ultrasound haptics. IEEE transactions on haptics 13, 1 (2020), 87–93.
[42]
Nikolaus F Troje. 2002. Decomposing biological motion: A framework for analysis and synthesis of human gait patterns. Journal of vision 2, 5 (2002), 2–2.
[43]
Yangbo Xie, Chen Shen, Wenqi Wang, Junfei Li, Dingjie Suo, Bogdan-Ioan Popa, Yun Jing, and Steven A Cummer. 2016. Acoustic holographic rendering with two-dimensional metamaterial-based passive phased array. Scientific reports 6, 1 (2016), 35437.

Recommendations

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences
SIGGRAPH '24: ACM SIGGRAPH 2024 Conference Papers
July 2024
1106 pages
ISBN:9798400705250
DOI:10.1145/3641519
This work is licensed under a Creative Commons Attribution International 4.0 License.

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 13 July 2024

Check for updates

Author Tags

  1. Phase retrieval
  2. acoustic levitation
  3. computer generated holography
  4. particle displays

Qualifiers

  • Research-article
  • Research
  • Refereed limited

Funding Sources

  • EPSRC Prosperity Partnership
  • UKRI Frontier Research Guarantee Grant
  • U-H2020 through their ERC Advanced Grant
  • Royal Academy of Engineering through their Chairs in Emerging Technology Program

Conference

SIGGRAPH '24
Sponsor:

Acceptance Rates

Overall Acceptance Rate 1,822 of 8,601 submissions, 21%

Contributors

Other Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

  • 0
    Total Citations
  • 538
    Total Downloads
  • Downloads (Last 12 months)538
  • Downloads (Last 6 weeks)100
Reflects downloads up to 17 Jan 2025

Other Metrics

Citations

View Options

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

HTML Format

View this article in HTML Format.

HTML Format

Login options

Media

Figures

Other

Tables

Share

Share

Share this Publication link

Share on social media