Abstract
Currently, with the rapid development of industry, the amount of data generated by industrial enterprises and industrial business website is exponential growth, and the big data has different types. In this paper, we design and implement an industrial big data information acquisition and query system. The system is based on big data acquisition and analysis of industry news data and industrial products data. We use a template based information acquisition method to crawl data from industry related news data and industry products data. We also discuss the query performance of text industry data with text index and only by SQL without index. The system is useful for analysis the hot news in industrial field and industry public opinion, and it is also useful for providing reference and rapid search and comparison of the relevant industrial products price, inventory and other information.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Viktor, M.S., Kenneth, C.: Big Data: A Revolution That Will Trans-Form How We Live, Work, and Think. Houghton Mifflin Harcourt, Boston (2013)
Jsoup Open Source Project Distributed under the Liberal MIT License. http://jsoup.org/
Wang, J., Wu, J., Zhang, Y., He, G.: Content information extraction of theme web pages based on tag information. In: 7th IEEE International Symposium on Computational Intelligence and Design, pp. 501–504. IEEE Press, Los Alamitos, CA (2015)
He, G., Wang, J., Zhang, Y., Peng, Y.: Keyword extraction of web pages based on domain thesaurus. In: 3th IEEE International Conference on Cloud Computing and Intelligence Systems, pp. 310–315. IEEE Press, Los Alamitos, CA (2014)
Theobald, M., Schenkel, R., Weikum, G.: Classification and focused crawling for semistructured data. In: Blanken, H., Grabs, T., Schek, H.-J., Schenkel, R., Weikum, G. (eds.) Intelligent Search on XML Data. LNCS, vol. 2818, pp. 145–157. Springer, Heidelberg (2003). doi:10.1007/978-3-540-45194-5_10
Wang, J., Yang, S., Wang, Y., Han, C.: The crawling and analysis of agricultural products big data based on Jsoup. In: 12th IEEE International Conference on Fuzzy Systems and Knowledge Discovery, pp. 1231–1236. IEEE Press, Los Alamitos, CA (2016)
Bootstrap Front-end Frameworks and Open Source Projects licensed by MIT. http://getbootstrap.com/
Jia, M., Xu, H., Wang, J., Bai, Y., Liu, B., Wang, J.: Handling big data of online social networks on a small machine. J. Comput. Soc. Netw. 2(1), 1–12 (2015)
Acknowledgments
This work was supported by the Key Laboratory of machine intelligence and advanced computing (MSC-201707A); Capital Normal University interdisciplinary research project; Capital Normal University science and technology innovation platform project.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2017 Springer International Publishing AG
About this paper
Cite this paper
Wang, J., Peng, Y., Lin, Y., Wang, K. (2017). Template Based Industrial Big Data Information Extraction and Query System. In: Tan, Y., Takagi, H., Shi, Y. (eds) Data Mining and Big Data. DMBD 2017. Lecture Notes in Computer Science(), vol 10387. Springer, Cham. https://doi.org/10.1007/978-3-319-61845-6_25
Download citation
DOI: https://doi.org/10.1007/978-3-319-61845-6_25
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-61844-9
Online ISBN: 978-3-319-61845-6
eBook Packages: Computer ScienceComputer Science (R0)