Van-Quyet Nguyen
ABSTRACT
Nowadays, big data enables to discover many aspects in agriculture sector such as finding unknown crop patterns or predicting the price of products. However, these massive data are often complex and heterogeneous which includes both structured (e.g., farm information) and unstructured data (e.g., image data, sensor data). It is required new techniques and tools to extract and represent valuable information in the form of human understanding to improve decision making for enhancing farm management. In this paper, we propose a big data knowledge acquisition platform which consists of efficient knowledge acquisition techniques integrated with an intuitive visualization tool supporting decision making applications. Firstly, we deploy open source big data frameworks (e.g., Flume, Hive, HBase) to support developing of multiple methods for collecting and storing data. Secondly, we implement distributed machine learning techniques on Hadoop and Spark to acquire knowledge from big data sources. Finally, we provide a visualization tool on web interface which can display extracted knowledge in multiple views (e.g., charts, tables) to support decision making applications. Experiments with real datasets show that the proposed platform is efficient and effective to answer important questions in smart farming.
- Rajeev Agrawal, Anirudh Kadadi, Xiangfeng Dai, and Frederic Andres. 2015. Challenges and opportunities with big data visualization. In Proceedings of the 7th International Conference on Management of computational and collective intElligence in Digital EcoSystems. ACM, 169–173.Google Scholar
Digital Library
- Rakesh Agrawal, Ramakrishnan Srikant, 1994. Fast algorithms for mining association rules. In Proc. 20th int. conf. very large data bases, VLDB, Vol. 1215. 487–499.Google Scholar
- G Amir and H Murtaza. 2015. Big data concepts, methods and analytics. International Journal of Information Management 35 (2015), 140.Google Scholar
- Thomas Bräunl, Stefan Feyrer, Wolfgang Rapf, and Michael Reinhardt. 2013. Parallel image processing. Springer Science & Business Media.Google Scholar
- Shuangxi Chen, Chunming Wu, and Yongmao Yu. 2016. Analysis of plant breeding on hadoop and spark. Advances in Agriculture 2016 (2016).Google Scholar
- Jeffrey Dean and Sanjay Ghemawat. 2008. MapReduce: simplified data processing on large clusters. Commun. ACM 51, 1 (2008), 107–113.Google ScholarDigital Library
- Olakunle Elijah, Tharek Abdul Rahman, Igbafe Orikumhi, Chee Yen Leow, and MHD Nour Hindia. 2018. An overview of Internet of things (IoT) and data analytics in agriculture: Benefits and challenges. IEEE Internet of Things Journal 5, 5 (2018), 3758–3773.Google ScholarCross Ref
- Jonathon S Hare, Sina Samangooei, and David P Dupplaw. 2011. OpenIMAJ and ImageTerrier: Java libraries and tools for scalable multimedia analysis and indexing of images. In Proceedings of the 19th ACM international conference on Multimedia. ACM, 691–694.Google ScholarDigital Library
- Abdulnasser Hatemi-J. 2004. Multivariate tests for autocorrelation in the stable and unstable VAR models. Economic Modelling 21, 4 (July 2004), 661–683. https://ideas.repec.org/a/eee/ecmode/v21y2004i4p661-683.htmlGoogle Scholar
- Tao Li, Xueyu Li, and Xu Zhang. 2017. The Design and Implementation of Vector Autoregressive Model and Structural Vector Autoregressive Model Based on Spark. In 2017 3rd International Conference on Big Data Computing and Communications (BIGCOM). IEEE, 386–394.Google Scholar
- Zhicheng Liu, Biye Jiang, and Jeffrey Heer. 2013. imMens: Real-time Visual Querying of Big Data. In Computer Graphics Forum, Vol. 32. Wiley Online Library, 421–430.Google Scholar
- Van-Quyet Nguyen, Sinh Ngoc Nguyen, and Kyungbaek Kim. 2017. Design of a platform for collecting and analyzing agricultural big data. Journal of Digital Contents Society 18, 1 (2017), 149–158.Google ScholarCross Ref
- Cristina Nicolescu and Pieter Jonker. 2000. Parallel low-level image processing on a distributed-memory system. In International Parallel and Distributed Processing Symposium. Springer, 226–233.Google ScholarCross Ref
- S Rajeswari, K Suthendran, and K Rajakumar. 2017. A smart agricultural model by integrating IoT, mobile and cloud-based big data analytics. In 2017 International Conference on Intelligent Computing and Control (I2C2). IEEE, 1–5.Google ScholarCross Ref
- Partha Pratim Ray. 2017. Internet of things for smart agriculture: Technologies, practices and future direction. Journal of Ambient Intelligence and Smart Environments 9, 4(2017), 395–420.Google ScholarDigital Library
- Ashok Savasere, Edward Robert Omiecinski, and Shamkant B Navathe. 1995. An efficient algorithm for mining association rules in large databases. Technical Report. Georgia Institute of Technology.Google Scholar
- Purnima Shah, Deepak Hiremath, and Sanjay Chaudhary. 2017. Towards development of spark based agricultural information system including geo-spatial data. In 2017 IEEE International Conference on Big Data (Big Data). IEEE, 3476–3481.Google ScholarCross Ref
- Andrey Sozykin and Timofei Epanchintsev. 2015. MIPr-a framework for distributed image processing using Hadoop. In 2015 9th International Conference on Application of Information and Communication Technologies (AICT). IEEE, 35–39.Google ScholarCross Ref
- Venkataramana Sucharitha, SR Subash, and P Prakash. 2014. Visualization of big data: its tools and challenges. International Journal of Applied Engineering Research 9, 18 (2014), 5277–5290.Google Scholar
- Chris Sweeney, Liu Liu, Sean Arietta, and Jason Lawrence. 2011. HIPI: a Hadoop image processing interface for image-based mapreduce tasks. Chris. university of Virginia 2, 1 (2011), 1–5.Google Scholar
- Kindie Tesfaye, Kai Sonder, J Caims, Cosmos Magorokosho, Amsal Tarekegn, Girma T Kassie, Fite Getaneh, Tahirou Abdoulaye, Tsedeke Abate, and Olaf Erenstein. 2016. Targeting drought-tolerant maize varieties in southern Africa: a geospatial crop modeling approach using big data. (2016).Google Scholar
- Anup Vibhute and SK Bodhe. 2012. Applications of image processing in agriculture: a survey. International Journal of Computer Applications 52, 2(2012).Google ScholarCross Ref
- Jie Wang, Shuo Yang, Yuezhi Wang, and Cheng Han. 2015. The crawling and analysis of agricultural products big data based on Jsoup. In 2015 12th International Conference on Fuzzy Systems and Knowledge Discovery (FSKD). IEEE, 1197–1202.Google Scholar
- Lidong Wang, Guanghui Wang, and Cheryl Ann Alexander. 2015. Big data and visualization: methods, challenges and technology progress. Digital Technologies 1, 1 (2015), 33–38.Google Scholar
- Matei Zaharia, Mosharaf Chowdhury, Michael J Franklin, Scott Shenker, and Ion Stoica. 2010. Spark: Cluster computing with working sets.HotCloud 10, 10-10 (2010), 95.Google ScholarDigital Library
- Ji-chun Zhao and Jian-xin Guo. 2018. Big data analysis technology application in agricultural intelligence decision system. In 2018 IEEE 3rd International Conference on Cloud Computing and Big Data Analysis (ICCCBDA). IEEE, 209–212.Google Scholar
Index Terms
- Big Data Knowledge Acquisition Platform for Smart Farming
Recommendations
Design and Implementation of Componentized Big Data Platform
RICAI '20: Proceedings of the 2020 2nd International Conference on Robotics, Intelligent Control and Artificial IntelligenceThe rapid development of the Internet has produced massive amounts of data, and has promoted the development of various big data components in the big data field, such as Hadoop, Spark and other big data components. However, many big data components ...
Parallel rough set based knowledge acquisition using MapReduce from big data
BigMine '12: Proceedings of the 1st International Workshop on Big Data, Streams and Heterogeneous Source Mining: Algorithms, Systems, Programming Models and ApplicationsNowadays, with the volume of data growing at an unprecedented rate, big data mining and knowledge discovery have become a new challenge. Rough set theory for knowledge acquisition has been successfully applied in data mining. The recently introduced ...
Research on Big Data Parallel Processing Platform Based on Postal Industry
ICDSP '20: Proceedings of the 2020 4th International Conference on Digital Signal ProcessingWith the development of cloud computing, big data, and the Internet of Things, for the data collection and daily drama of the postal express industry, in the face of such a large-scale data set, the traditional storage and calculation related theories ...
Comments