Abstract
Viewpoint is vital in guiding the user to understand the volume data. However, a model that can recommend viewpoints conforming to user preference is hard to be represented explicitly. In this work, we propose an implicit model for the best viewpoint recommendation of volume visualization with CNN-based models to learn the traditional scoring method and user preference. Residual structures are applied for reducing overfitting in simple scalar regression and solving the problem of accuracy getting lower as the network getting deeper. Multi-level-based structures are applied to imitate the coarse and fine level in human perception. The detailed experiments of comparison between our model and traditional methods confirm the efficiency of our work. A case of application verifies that our model can flexibly realize a user preference-based best viewpoint selection in volume visualization.
Graphic abstract
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Behera L, Kumar S, Patnaik A (2006) On adaptive learning rate that guarantees convergence in feedforward networks. IEEE Trans Neural Netw 17(5):1116–1125
Bianco S, Celona L, Napoletano P, Schettini R (2016) Predicting image aesthetics with deep learning. In: International Conference on advanced concepts for intelligent vision systems, pp 117–125. Springer
Bordoloi UD, Shen H-W (2005) View selection for volume rendering. In: Visualization, 2005. VIS 05. IEEE, pp 487–494. IEEE
Dalal N, Triggs B (2005) Histograms of oriented gradients for human detection. In: IEEE computer society conference on computer vision & pattern recognition, pp 886–893
Guralnik G, Zemach C, Warnock T (1985) An algorithm for uniform random sampling of points in and on a hypersphere. Inf Process Lett 21(1):17–21
Hawkins DM (2004) The problem of overfitting. J Chem Inf Comput Sci 44(1):1–12
He K, Zhang X, Ren S, Sun J (2015) Spatial pyramid pooling in deep convolutional networks for visual recognition. IEEE Trans Pattern Anal Mach Intell 37(9):1904–1916
He K, Zhang X, Ren S, Sun J (2016) Deep residual learning for image recognition. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 770–778, 2016
Hicks J, Wheeling R (1959) An efficient method for generating uniformly distributed points on the surface of an n-dimensional sphere. Commun ACM 2(4):17–19
Ioffe S, Szegedy C (2015) Batch normalization: accelerating deep network training by reducing internal covariate shift. arXiv preprint arXiv:1502.03167
Ji G, Shen H-W (2006) Dynamic view selection for time-varying volumes. IEEE Trans Vis Comput Graph 12(5):1109–1116
Karp RM (1972) Reducibility among combinatorial problems. In: Complexity of computer computations. Springer, pp 85–103
LeCun Y, Bottou L, Bengio Y, Haffner P (1998) Gradient-based learning applied to document recognition. Proc IEEE 86(11):2278–2324
Levin DT, Simons DJ (1997) Failure to detect changes to attended objects in motion pictures. Psychon Bull Rev 4(4):501–506
Lin M, Chen Q, Yan S (2013) Network in network. arXiv preprint arXiv:1312.4400
Lindemann F, Ropinski T (2011) About the influence of illumination models on image comprehension in direct volume rendering. IEEE Trans Vis Comput Graph 17(12):1922–1931
Long J, Shelhamer E, Darrell T (2015) Fully convolutional networks for semantic segmentation. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 3431–3440
Lowe DG (2004) Distinctive image features from scale-invariant keypoints. Int J Comput Vis 60(2):91–110
Lu X, Lin Z, Jin H, Yang J, Wang JZ (2015) Rating image aesthetics using deep learning. IEEE Trans Multimed 17(11):2021–2034
Mai L, Jin H, Liu F (2016) Composition-preserving deep photo aesthetics assessment. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 497–506
Malu G, Bapi RS, Indurkhya B (2017) Learning photography aesthetics with deep CNNS. arXiv preprint arXiv:1707.03981,
Marsaglia G (2003) Seeds for random number generators. Commun ACM 46(5):90–93
Ojala T, Pietikainen M, Maenpaa T (2002) Multiresolution gray-scale and rotation invariant texture classification with local binary patterns. IEEE Trans Pattern Anal Mach Intell 24(7):971–987
Secord A, Lu J, Finkelstein A, Singh M, Nealen A (2011) Perceptual models of viewpoint preference. ACM Trans Graph (TOG) 30(5):109
Shen H-W, Johnson CR (1995) Sweeping simplices: a fast iso-surface extraction algorithm for unstructured grids. In: Proceedings of the 6th conference on Visualization’95, p 143. IEEE computer society
Shi N, Tao Y (2019) Cnns based viewpoint estimation for volume visualization. ACM Trans Intell Syst Technol (TIST) 10(3):27
Srivastava RK, Greff K, Schmidhuber J (2015) Highway networks. arXiv preprint arxiv:1505.00387
Stanley RP (1975) The Fibonacci lattice. Fibonacci Q 13(3):215–232
Takahashi S, Fujishiro I, Takeshima Y, Nishita T (2005) A feature-driven approach to locating optimal viewpoints for volume visualization. In: Visualization, 2005. VIS 05. IEEE, pp 495–502. IEEE
Tao Y, Wang Q, Chen W, Wu Y, Lin H (2016) Similarity voting based viewpoint selection for volumes. In: Computer graphics forum, vol 35. Wiley, pp 391–400
Vázquez P-P, Feixas M, Sbert M, Heidrich W (2001) Viewpoint selection using viewpoint entropy. VMV 1:273–280
Viola I, Feixas M, Sbert M, Groller ME (2006) Importance-driven focus of attention. IEEE Trans Vis Comput Graph 12(5):933–940
Viola I, Kanitsar A, Gröller ME (2005) Importance-driven feature enhancement in volume visualization. IEEE Trans Vis Comput Graph 11(4):408–418
Weaver W (1949) The mathematics of communication. Sci Am 181(1):11–15
Zhao H, Shi J, Qi X, Wang X, Jia J (2017) Pyramid scene parsing network. In: IEEE Conference on computer vision and pattern recognition (CVPR), pp 2881–2890
Zheng Z, Ahmed N, Mueller K (2011) iView: a feature clustering framework for suggesting informative views in volume visualization. IEEE Trans Vis Comput Graph 17(12):1959–1968
Acknowledgements
This work is supported by the National Key Research and Development Program of China (2016QY02D0304), NSFC No. 61672055, and the National Program on Key Basic Research Project (973 Program) No. 2015CB352503.
Author information
Authors and Affiliations
Corresponding author
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
About this article
Cite this article
Yang, C., Li, Y., Liu, C. et al. Deep learning-based viewpoint recommendation in volume visualization. J Vis 22, 991–1003 (2019). https://doi.org/10.1007/s12650-019-00583-4
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s12650-019-00583-4