Abstract
The data analysis software of the Fish4Knowledge (F4K) system is composed of several image processing modules and the execution of the processes is conducted by a workflow engine. As image processing is a data-intensive computing task, it often involves many computational processes and frequent data transfer. Traditional high performance computing platforms usually require special applications to move data from storage facilities to the computing nodes, and data transfer can become the main bottleneck of the whole process workflow as the applications scale up. In this study, we adopted the concept of cloud computing and implemented a computing platform which combined virtualization and distributed computing technologies, to support image data analysis on a large scale. A uniform job execution interface was also developed to hide the complexity of resource assembly from the users and to simplify computing resource allocation and process execution.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
ALPS\(\_\)team. 2014. Alps. http://www.nchc.org.tw/en/services/supercomputing/supercomputing_1/ar585f1.php. Accessed 16 Nov 2014.
Apache. 2014a. Hadoop. http://hadoop.apache.org/. Accessed 16 Nov 2014.
Apache. 2014b. Storm. https://storm.apache.org/. Accessed 16 Nov 2014.
Chou, H., Y. Shiau, S. Lo, S. Lin, and F. Lin. 2009. A real-time ecological observation video streaming system based on grid architecture. In Proceeding HPC Asia (KaoShiung, Taiwan)
Chung, T.-L., W.-Y. Chang, W.-F. Tsai, F.-P. Lin, E. Strandell, L.-C. Ku, J.-G. Lee, J.-Y. Chang, T.-H. Lee, J.-H. Wu, S.-C. Lin, M. Chen, Y.-H. Lee, K.-C. Chang, and Y.-F. Wang. 2010. Cyberinfrastructure for flood mitigation in taiwan. Proceedings of the ICE—Water Management 163(1): 3–11.
DRMMA\(\_\)Working\(\_\)Group. 2014. Open grid forum drmma api. http://www.drmaa.org/index.php. Accessed 16 Nov 2014.
Grid\(\_\)Engine\(\_\)team. 2014. Grid engine. http://gridscheduler.sourceforge.net/. Accessed 16 Nov 2014.
IBM\(\_\)team. 2014. Ibm platform lsf. http://www2.nchc.org.tw/~a00yys00/lsf7/7.0.6/lsf_hpc_using/intro.html. Accessed 16 Nov 2014.
Lin, N.-C., T.-Y. Fan, F.-P. Lin, K.-T. Shao, and T.-H. Sheen. 2009. Monitoring of coral reefs at the intake inlet and outlet bay of the 3rd nuclear power plant in southern Taiwan. In Proceeding of annual meeting of the fisheries society of Taiwan.
Tsai, W.-F., W. Huang, F.-P. Lin, B. Hung, Y.-T. Wang, S. Shiau, S.-C. Lin, C.-H. Hsieh, H.-E. Yu, L.-L. Pan, and C.-L. Huang. 2008. The human-centered cyberinfratructure for scientific and engineering grid applications. Journal of the Chinese Institute of Engineers 31(7): 1127–1139.
Wu, J.-H., F.-P. Lin, and W.-F. Tsai. 2009. A scalable middleware for multimedia streaming. In Proceeding of HPC Asia 2009, Kaohsiung, Taiwan.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2016 Springer International Publishing Switzerland
About this chapter
Cite this chapter
Chang, JS., Lin, SI., Lin, FP., Chou, HM. (2016). Supercomputing Resources. In: Fisher, R., Chen-Burger, YH., Giordano, D., Hardman, L., Lin, FP. (eds) Fish4Knowledge: Collecting and Analyzing Massive Coral Reef Fish Video Data. Intelligent Systems Reference Library, vol 104. Springer, Cham. https://doi.org/10.1007/978-3-319-30208-9_3
Download citation
DOI: https://doi.org/10.1007/978-3-319-30208-9_3
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-30206-5
Online ISBN: 978-3-319-30208-9
eBook Packages: EngineeringEngineering (R0)