Skip to main content
SpringerLink
Log in

A 3D Shape Retrieval Framework Supporting Multimodal Queries

  • Published:
International Journal of Computer Vision Aims and scope Submit manuscript

Abstract

This paper presents a unified framework for 3D shape retrieval. The method supports multimodal queries (2D images, sketches, 3D objects) by introducing a novel view-based approach able to handle the different types of multimedia data. More specifically, a set of 2D images (multi-views) are automatically generated from a 3D object, by taking views from uniformly distributed viewpoints. For each image, a set of 2D rotation-invariant shape descriptors is produced. The global shape similarity between two 3D models is achieved by applying a novel matching scheme, which effectively combines the information extracted from the multi-view representation. The experimental results prove that the proposed method demonstrates superior performance over other well-known state-of-the-art approaches.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  • Akgul, C., Axenopoulos, A., Bustos, B., Chaouch, M., Daras, P., Dutagaci, H., Furuya, T., Godil, A., Kreft, S., Lian, Z., Napoleon, T., Mademlis, A., Ohbuchi, R., Rosin, P. L., Sankur, B., Schreck, T., Sun, X., Tezuka, M., Yemez, Y., Verroust-Blondet, A., & Walter, M. (2009). SHREC 2009—generic shape retrieval contest. In 30th international conference on EUROGRAPHICS 2009, workshop on 3D object retrieval. Munich, Germany, March 2009.

  • Ankerst, M., Kastenmuller, G., Kriegel, H. P., & Seidl, T. (1999). 3D shape histograms for similarity search and classification in spatial databases. In Proc. of the 6th int. symp. spatial databases (SSD1999). Hong Kong, 1999.

  • Ansary, T. F., Vandeborre, J.-P., & Daoudi, M. (2007). 3D-model search engine from photos. In Proc. ACM CIVR 2008 (pp. 89–92).

  • Axenopoulos, A., Daras, P., Dutagaci, H., Furuya, T., Godil, A., & Ohbuchi, R. (2009). SHREC 2009—shape retrieval contest of partial 3D models. In 30th international conference on EUROGRAPHICS 2009, workshop on 3D object retrieval. Munich, Germany, March 2009.

  • Belkasim, S. O., Shridhar, M., & Ahmadi, M. (1991). Pattern recognition with moment invariants: A comparative study and new results. Pattern Recognition, 24(12), 1117–1138.

    Article  Google Scholar 

  • Bustos, B., Keim, D. A., Saupe, D., Schreck, T., & Vranic, D. V. (2005). Feature-based similarity search in 3D object databases. ACM Computing Surveys, 37(4), 345–387.

    Article  Google Scholar 

  • Bustos, B., Keim, D., Saupe, D., & Schreck, T. (2007). Content-based 3d object retrieval. IEEE Computer Graphics and Applications, 27(4), 22–27.

    Article  Google Scholar 

  • Canterakis, N. (1999). 3D Zernike moments and Zernike affine invariants for 3D image analysis and recognition. In Proc. of the Scandinavian conference on image analysis, 1999.

  • Chaouch, M., & Verroust-Blondet, A. (2006). Enhanced 2D/3D approaches based on relevance index for 3D-shape retrieval. In SMI’06. Matsushima, Japan, June 2006.

  • Chen, D.-Y., & Ouhyoung, M. (2002). A 3D object retrieval system based on multi-resolution reeb graph. In Proc. of Computer Graphics Workshop. Tainan, ROC.

  • Chen, D.-Y., Ouhyoung, M., Tian, X.-P., Shen, Y.-T., & Ouhyoung, M. (2003). On visual similarity based 3d model retrieval. In Proc. of Eurographics (pp. 223–232). Granada, Spain.

  • Daras, P., Zarpalas, D., Axenopoulos, A., Tzovaras, D., & Strintzis, M. G. (2006a). Three-dimensional shape-structure comparison method for protein classification. IEEE/ACM Transactions on Computational Biology and Bioinformatics, 3(3), 193–207.

    Article  Google Scholar 

  • Daras, P., Zarpalas, D., Tzovaras, D., & Strintzis, M. G. (2006b). Efficient 3-d model search and retrieval using generalized 3-d radon transforms. IEEE Transactions on Multimedia, 8(1), 101–114.

    Article  Google Scholar 

  • Filali Ansary, T., Daoudi, M., & Vandeborre, J.-P. (2007). A Bayesian 3D search engine using adaptive views clustering. IEEE Transactions on Multimedia, 9(1), 78–88.

    Article  Google Scholar 

  • Fischer, K., & Gartner, B. (2004). The smallest enclosing ball of balls: Combinatorial structure and algorithms. International Journal of Computational Geometry and Applications (IJCGA), 14, 341–387.

    Article  MATH  MathSciNet  Google Scholar 

  • Fodor, J. A. (1983). The modularity of mind. Cambridge: Cambridge Bradford Books, MIT Press.

    Google Scholar 

  • Funkhouser, T., Min, P., Kazhdan, M., Chen, J., Halderman, A., Dobkin, D., & Jacobs, D. (2003). A search engine for 3D models. ACM Transactions on Graphics, 22(1), 83–105.

    Article  Google Scholar 

  • Goodall, S., Lewis, P. H., Martinez, K., Sinclair, P. A. S., Giorgini, F., Addis, M., Boniface, M. J., Lahanier, C., & Stevenson, J. (2004). SCULPTEUR: Multimedia retrieval for museums. In Proc. of the image and video retrieval: Third international conference (CIVR’04) (pp. 638–646).

  • Hartveldt, J., Spagnuolo, M., Axenopoulos, A., Biasotti, S., Daras, P., Dutagaci, H., Furuya, T., Godil, A., Li, X., Mademlis, A., Marini, S., Napoleon, T., Ohbuchi, R., & Tezuka, M. (2009). SHREC 2009 track: Structural shape retrieval on watertight models. In 30th international conference on EUROGRAPHICS 2009, workshop on 3D object retrieval. Munich, Germany, March 2009.

  • Hilaga, M., Shinagawa, Y., Kohmura, T., & Kunii, T. L. (2001). Topology matching for fully automatic similarity estimation of 3D shapes. In Proc. of ACM SIGGRAPH 2001 (pp. 203–212).

  • Horn, B. (1984). Extended Gaussian images. Proceedings of the IEEE, 72(12), 1671–1686.

    Article  Google Scholar 

  • Hu, M. K. (1962). Visual pattern recognition by moment invariants. IRE Transactions on Information Theory, 8, 179–197.

    Google Scholar 

  • Huber, F., & Hebert, M. (2001). Fully automatic registration of multiple 3D data sets. In Proc. of IEEE workshop on computer vision beyond the visible spectrum: Methods and applications (CVBVS). Kauai, Hawaii, USA, Dec. 2001.

  • Iyer, M., Jayanti, S., Lou, K., Kalyanaraman, Y., & Ramani, K. (2005). Three dimensional shape searching: State-of-the-art review and future trends. Computer Aided Design, 5(15), 509–530.

    Article  Google Scholar 

  • Jayanti, S., Kalyanaraman, K., Iyer, N., & Ramani, K. (2006). Developing an engineering shape benchmark for CAD models. Computer-Aided Design, 38(9), 939–953.

    Article  Google Scholar 

  • Kang, S. B., & Ikeuchi, K. (1993). The complex EGI: A new representation for 3D pose determination. IEEE Transactions on Pattern Analysis and Machine Intelligence, 15(7), 707–721.

    Article  Google Scholar 

  • Katz, S., & Tal, A. (2003). Hierarchical mesh decomposition using fuzzy clustering and cuts. ACM Transactions on Graphics 954–961.

  • Kazhdan, M., Funkhouser, T., & Rusinkiewicz, S. (2003). Rotation invariant spherical harmonic representation of 3D shape descriptors. In Proc. of symposium on geometry processing, Jun. 2003.

  • Khotanzad, A., & Hong, Y. H. (1990). Invariant image recognition by Zernike moments. IEEE Transactions on Pattern Analysis and Machine Intelligence, 12(5), 489–497.

    Article  Google Scholar 

  • Kriegel, H.-P., Kroeger, P., Mashael, Z., Pfeifle, M., Poetke, M., & Seidl, T. (2003). Effective similarity search on voxelized cad objects. In Proc. of IEEE eighth international conference on database systems for advanced applications. Washington, DC, USA.

  • Lindstrom, P., & Turk, G. (2000). Image-driven simplification. ACM Transactions on Graphics, 19(3), 204–241.

    Article  Google Scholar 

  • Liu, Y., Zha, H., & Qin, H. (2006). The generalized shape distributions for shape matching and analysis. In Proc. of the IEEE international conference on shape modeling and applications (SMI2006). Matsushima, Japan.

  • Lowe, D. G. (2004). Distinctive image features from scale-invariant keypoints. International Journal of Computer Vision, 60(2).

  • Mademlis, A., Daras, P., Axenopoulos, A., Tzovaras, D., & Strintzis, M. G. (2008a). Combining topological and geometrical features for global and partial 3D shape retrieval. IEEE Transactions on Multimedia, 10(5), 819–831.

    Article  Google Scholar 

  • Mademlis, A., Daras, P., Tzovaras, D., & Strintzis, M. G. (2008b). 3D object retrieval based on resulting fields. In 29th international conference on EUROGRAPHICS 2008, workshop on 3D object retrieval. Crete, Greece, April 2008.

  • Mahmoudi, S., & Daoudi, M. (2002). 3D models retrieval by using characteristic views. In ICPR’02 (pp. 11–15). Quebec, Canada, Aug. 2002.

  • Napoleon, T., Adamek, T., Shmitt, F., & O’Connor, N. E. (2008). SHREC’08 entry: Multi-view 3D retrieval using multi-scale contour representation. In Proc. IEEE SMI 2009.

  • Novotni, M., & Klein, R. (2003). 3d Zernike descriptors for content based shape retrieval. In Proc. of the eighth ACM symposium on solid modeling and applications (pp. 216–225). NY, USA, 2003. New York: ACM.

    Chapter  Google Scholar 

  • Ohbuchi, R., Otagiri, T., Ibato, M., & Takei, T. (2002). Shape-similarity search of three-dimensional models using parameterized statistics. In Oroc. of Pacific graphics (pp. 265–274). Beijng, China. Los Alamitos: IEEE Comput. Soc.

    Google Scholar 

  • Ohbuchi, R., Minamitani, T., & Takei, T. (2003a). Shape similarity search of 3D models by using enhanced shape functions. In Proc. TP.CG. 03 (pp. 97–104).

  • Ohbuchi, R., Nakazawa, M., & Takei, T. (2003b). Retrieving 3D shapes based on their appearance. In Proc. ACM workshop on multimedia information retrieval (MIR) 2003 (pp. 39–46).

  • Ohbuchi, R., Osada, K., Furuya, T., & Banno, T. (2008). Salient local visual features for shape-based 3D model retrieval. In Proc. of the IEEE international conference on shape modeling and applications (SMI 2008) (pp. 93–102).

  • Osada, R., Funkhouser, T., Chazelle, B., & Dobkin, D. (2001). Matching 3D models with shape distributions. In Proc. of the IEEE international conference on shape modeling and applications (SMI2001) (pp. 154–166).

  • Osada, R., Funkhouser, T., Chazelle, B., & Dobkin, D. (2002). Shape distributions. ACM Transactions on Graphics, 21(4), 807–832.

    Article  Google Scholar 

  • Papadakis, P., Pratikakis, I., Perantonis, S., & Theoharis, T. (2007). Efficient 3D shape matching and retrieval using a concrete radialized spherical projection representation. Pattern Recognition, 40(9), 2437–2452.

    Article  MATH  Google Scholar 

  • Pu, J., & Ramani, K. (2005). An approach to drawing-like view generation from 3D models. In Proc. of IDETC/CIE 2005. ASME.

  • Real-time 3D models, http://www.3drt.com/.

  • Shih, J.-L., Lee, C.-H., & Wang, J. T. (2007). A new 3D model retrieval approach based on the elevation descriptor. Pattern Recognition, 40(1), 283–295.

    Article  MATH  Google Scholar 

  • Shilane, P., Min, P., Kazhdan, M., & Funkhouser, T. (2004). The Princeton shape benchmark. In Proceedings of the shape modeling international (SMI ’04) (pp. 167–178). Genova, Italy, June 2004.

  • Tal, A., & Zuckerberger, E. (2006). Mesh retrieval by components. International Conference on Computer Graphics Theory and Applications, 142–149.

  • Tangelder, J., & Veltkamp, R. C. (2004). A survey of content based 3D shape retrieval methods. In Proc. of IEEE shape modelling international (pp. 145–156).

  • Teague, M. R. (1979). Image analysis via the general theory of moments. Journal of Optical Society of America, 70, 920–930.

    Article  MathSciNet  Google Scholar 

  • The VICTORY 3D Search Engine, http://www.victory-eu.org:8080/victory/results/search.html.

  • Tsatsaias, V., Daras, P., & Strintzis, M. G. (2007). 3D protein classification using topological, geometrical and biological information. In Proc. of IEEE international conference on image processing (ICIP 2007). San Antonio, Texas, USA, 2007.

  • Tung, T., & Schmitt, F. (2004). Augmented reeb graphs for content-based retrieval of 3D mesh models. In Proc. of the IEEE international conference on shape modeling and applications (SMI2004) (pp. 157–166).

  • Tung, T., & Schmitt, F. (2005). The augmented multiresolution Reeb graph approach for content-based retrieval of 3D shapes. International Journal of Shape Modeling (IJSM), 11(1).

  • Vinanco, A. P., Ramirez, A. M., & Agustin, F. G. (2003). Digital image reconstruction by using Zernike moments. In Proc. of SPIE (pp. 281–289). Barcelona, Spain, Sept. 2003.

  • Vranic, D. V. (2003). An improvement of rotation invariant 3d-shape based on functions on concentric spheres. In Proc. of IEEE ICIP (3) (pp. 757–760).

  • Vranic, D. (2004). 3d model retrieval. Ph.D. Dissertation, University of Leipzig.

  • Vranic, D., & Saupe, D. (2002). Description of 3d-shape using a complex function on the sphere. In Proc. of the IEEE international conference on multimedia and Expo (ICME2002) (Vol. 1, pp. 177–180).

  • Vranic, D., Saupe, D., & Richter, J. (2001). Tools for 3d-object retrieval: Karhunen-loeve transform and spherical harmonics. In Proc. of the IEEE fourth workshop on multimedia signal processing (pp. 293–29).

  • Wahl, E., Hillenbrand, U., & Hirzinger, G. (2003). Surflet-pair-relation histograms: A statistical 3D-shape representation for rapid classification. In Proc. 3DIM 2003 (pp. 474–481).

  • Weyrich, T., Pauly, M., Keiser, R., Heinzle, S., Scandella, S., & Gross, M. (2004). Post-processing of scanned 3d surface data. In Proc. of Eurographics. Granada, Spain, 2004.

  • Yap, P. T., Paramesran, R., & Ong, S. H. (2003). Image analysis by Krawtchouk moments. IEEE Transactions on Image Processing 12(11), 1367–1377.

    Article  MathSciNet  Google Scholar 

  • Zarpalas, D., Daras, P., Axenopoulos, A., Tzovaras, D., & Strintzis, M. G. (2007). 3D model search and retrieval using the spherical trace transform. EURASIP Journal on Advances in Signal Processing. doi:10.1155/2007/23912. Volume 2007, Article ID 23912, 14 pages.

  • Zhang, D., & Lu, G. (2002). Shape-based image retrieval using generic Fourier descriptor. ELSEVIER Signal Processing: Image Communication, 17(10), 825–848.

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Informatics and Telematics Institute, Centre for Research and Technology Hellas, Thessaloniki, Greece

    Petros Daras & Apostolos Axenopoulos

Authors
  1. Petros Daras
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Apostolos Axenopoulos
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Petros Daras.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Daras, P., Axenopoulos, A. A 3D Shape Retrieval Framework Supporting Multimodal Queries. Int J Comput Vis 89, 229–247 (2010). https://doi.org/10.1007/s11263-009-0277-2

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11263-009-0277-2

Keywords

Search

Navigation