Skip to main content
SpringerLink
Log in

A brush device with visual and haptic feedback for virtual painting of 3D virtual objects

  • Original Article
  • Published:
Virtual Reality Aims and scope Submit manuscript

Abstract

We have previously developed a mixed reality (MR) painting system with which a user could take a physical object in the real world and apply virtual paint to it. However, this system could not provide the sensation of painting on virtual objects in MR space. Therefore, we subsequently proposed and developed mechanisms that simulated the effect of touch and movement when a brush device was used to paint on a virtual canvas. In this paper, we use visual and haptic feedback to provide the sensation of painting on virtual three-dimensional objects using a new brush device called the MAI Painting Brush++. We evaluate and confirm its effectiveness through several user studies.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15
Fig. 16
Fig. 17
Fig. 18
Fig. 19
Fig. 20
Fig. 21
Fig. 22

Similar content being viewed by others

Notes

  1. “Sensu brush” http://www.sensubrush.com/.

References

  • Bau O, Poupyrev I, Le Goc M, Galliot L, Glisson M (2012) REVEL: tactile feedback technology for augmented reality. In: ACM SIGGRAPH 2012 emerging technologies (SIGGRAPH ‘12). ACM, New York, NY, USA, Article 17. doi: 10.1145/2343456.2343473

  • Baxter B, Scheib V, Lin MC, Manocha D (2001) DAB: interactive haptic painting with 3D virtual brushes. In: Proceedings of the 28th annual conference on Computer graphics and interactive techniques (SIGGRAPH ‘01). ACM, New York, NY, USA, pp 461–468. doi: 10.1145/383259.383313

  • Baxter W, Wendt J, Lin MC (2004) IMPaSTo: a realistic, interactive model for paint. In: Spencer SN (Ed.) Proceedings of the 3rd international symposium on non-photorealistic animation and rendering (NPAR ‘04). ACM, New York, NY, USA, pp 45–148. doi: 10.1145/987657.987665

  • Chu NS-H, Tai C-L (2005) MoXi: real-time ink dispersion in absorbent paper. In: J Buhler (Ed.) ACM SIGGRAPH 2005 sketches (SIGGRAPH ‘05). ACM, New York, NY, USA, Article 62. doi: 10.1145/1187112.1187186

  • Curtis CJ, Anderson SE, Seims JE, Fleischer KW, Salesin DH (1997) Computer-generated watercolor. In: Proceedings of the 24th annual conference on Computer graphics and interactive techniques (SIGGRAPH ‘97). ACM Press/Addison-Wesley Publishing Co., New York, NY, USA, pp 421–430. doi: 10.1145/258734.258896

  • Foskey M, Otaduy MA, Lin MC (2005) ArtNova: Touch-enabled 3D model design. In: J Fujii (Ed.) ACM SIGGRAPH 2005 courses (SIGGRAPH ‘05). ACM, New York, NY, USA, Article 188. doi: 10.1145/1198555.1198619

  • Gooch B, Gooch A (2001) Non-photorealistic rendering. A. K. Peters Ltd., Natick

    MATH  Google Scholar 

  • Iwai D, Sato K (2005) Heat sensation in image creation with thermal vision. In: Proceedings of the 2005 ACM SIGCHI international conference on advances in computer entertainment technology (ACE ‘05). ACM, New York, NY, USA, pp 213–216. doi: 10.1145/1178477.1178510

  • Kamuro S, Minamizawa K, Kawakami N, Tachi S (2009) Ungrounded kinesthetic pen for haptic interaction with virtual environments. In: Proceedings of robot and human interactive communication (RO-MAN 2009), pp 436–441, Sept. 27 2009–Oct. 2. doi: 10.1109/ROMAN.2009.5326217

  • Kamuro S, Minamizawa K, Tachi S (2011) 3D Haptic modeling system using ungrounded pen-shaped kinesthetic display. In: Proceedings of the 2011 IEEE virtual reality conference (VR ‘11). IEEE Computer Society, Washington, DC, USA, pp 217–218. doi: 10.1109/VR.2011.5759476

  • Kato G, Kuroda Y, Nisky I, Kiyokawa K, Takemura H (2015) HapSticks: tool-mediated interaction with grounding-free haptic interface. In: SIGGRAPH Asia 2015 haptic media and contents design (SA ‘15). ACM, New York, NY, USA, Article 7. doi: 10.1145/2818384.2818387

  • Kim H-J, Kim H, Chae S, Seo J, Han T-D (2013) AR pen and hand gestures: a new tool for pen drawings. In CHI ‘13 extended abstracts on human factors in computing systems (CHI EA ‘13). ACM, New York, NY, USA, pp 943–948. doi: 10.1145/2468356.2468525

  • Liu X, Gu J (2014) FlexStroke: a flexible, deformable brush-tip with dynamic stiffness for digital input. In: Proceedings of the 8th international conference on tangible, embedded and embodied interaction (TEI ‘14). ACM, New York, NY, USA, pp. 39–40. doi: 10.1145/2540930.2540982

  • Otsuki M, Sugihara K, Kimura A, Shibata F, Tamura H (2010) MAI painting brush: an interactive device that realizes the feeling of real painting. In: Proceedings of the 23nd annual ACM symposium on user interface software and technology (UIST ‘10). ACM, New York, NY, USA, pp 97–100. doi: 10.1145/1866029.1866045

  • Ryokai K, Marti S, Ishii H (2004) I/O brush: drawing with everyday objects as ink. In: Proceedings of the SIGCHI conference on human factors in computing systems (CHI ‘04). ACM, New York, NY, USA, pp 303–310. doi: 10.1145/985692.985731

  • Saito S, Nakajima M (1999) 3D physics-based brush model for painting. In: ACM SIGGRAPH 99 conference abstracts and applications (SIGGRAPH ‘99). ACM, New York, NY, USA, p 226. doi: 10.1145/311625.312110

  • Sandor C, Uchiyama S, Yamamoto H (2007) Visuo-haptic systems: half-mirrors considered harmful. In: Proceedings of the second joint EuroHaptics conference and symposium on haptic interfaces for virtual environment and teleoperator systems (WHC ‘07). IEEE Computer Society, Washington, DC, USA, pp 292–297. doi:10.1109/WHC.2007.125

  • Vandoren P, Claesen L, Van Laerhoven T, Taelman J, Raymaekers C, Flerackers E, Van Reeth F (2009) FluidPaint: an interactive digital painting system using real wet brushes. In: Proceedings of the ACM international conference on interactive tabletops and surfaces (ITS ‘09). ACM, New York, NY, USA, pp 53–56. doi: 10.1145/1731903.1731914

  • Yeom J, Lee G (2012) Designing a user interface for a painting application supporting real watercolor painting processes. In: Proceedings of the 10th Asia pacific conference on computer human interaction (APCHI ‘12). ACM, New York, NY, USA, pp 219–226. doi: 10.1145/2350046.2350091

Download references

Author information

Authors and Affiliations

  1. University of Tsukuba, 1-1-1 Tennodai, Tsukuba, Ibaraki, 305-8571, Japan

    Mai Otsuki

  2. Ritsumeikan University, 1-1-1 Noji-higashi, Kusatsu, Shiga, 525-8577, Japan

    Kenji Sugihara, Azusa Toda, Fumihisa Shibata & Asako Kimura

Authors
  1. Mai Otsuki
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Kenji Sugihara
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Azusa Toda
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Fumihisa Shibata
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Asako Kimura
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Mai Otsuki.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Otsuki, M., Sugihara, K., Toda, A. et al. A brush device with visual and haptic feedback for virtual painting of 3D virtual objects. Virtual Reality 22, 167–181 (2018). https://doi.org/10.1007/s10055-017-0317-0

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10055-017-0317-0

Keywords

Search

Navigation