Lili Wang
ABSTRACT
With the development of big data technology in the internet era, the recruitment industry also needs to apply big data technology to achieve data analysis tasks. This project designs and implements an automated recruitment data analysis system based on big data platform. The system uses big data technologies to accomplish data collection, storage, cleaning, analysis and visualization. In terms of data collection, distributed web crawler technology is used to ensure the efficiency of data acquisition. For data storage, a combination of distributed file system and relational database is used. In terms of data cleaning and analysis, recruitment data is collected and filtered through algorithms according to information rules from popular recruitment websites, providing basis for data visualization. Finally, multiple perspectives such as bar chart and heat map based on geographic location are used to visualize job positions' salary, requirements and demand levels, etc., providing big data analysis basis for job seekers and professionals.
- M. S. Sodhi and B. G. Son, ‘Content analysis of OR job advertisements to infer required skills’, J. Oper. Res. Soc., vol. 61, no. 9, pp. 1315–1327, Sep. 2010.Google Scholar
Cross Ref
- P. Motlogelwa, Z. A. Mbero, Y. Ayalew, T. Z. Nkgau, and A. Masizana-Katongo, ‘Computing knowledge and skills demand: A content analysis of job adverts in Botswana’, 2011, Accessed: Jul. 17, 2023.Google Scholar
- D. T Smith and A. Ali, ‘Analyzing Computer Programming Job Trend Using Web Data Mining’, Issues Informing Sci. Inf. Technol., vol. 11, pp. 203–214, 2014.Google ScholarCross Ref
- K. Song, H. Kim, J. Cha, and T. Lee, ‘Matching and Mismatching of Green Jobs: A Big Data Analysis of Job Recruiting and Searching’, Sustainability, vol. 13, no. 7, Art. no. 7, Jan. 2021.Google Scholar
- F. Ying and Z. Zhang, ‘Data Visualization Analysis of Big Data Recruitment Positions in Hangzhou Based on Python’, Rev. Comput. Eng. Stud., vol. 6, pp. 81–86, Dec. 2019.Google ScholarCross Ref
- Y. Chen, Y. Yi, and L. He, ‘Analysis on the Demand Characteristics of Tourism Talents Based on Recruitment Information of Three Recruitment Websites’, in Proceedings of the 2020 2nd International Conference on Big Data and Artificial Intelligence, Johannesburg South Africa: ACM, Apr. 2020, pp. 185–190.Google Scholar
- Y. Chen and R. Pan, ‘Research on Data Analysis and Visualization of Recruitment Positions Based on Text Mining’, Adv. Multimed, vol. 2022, p. e9047202, Jul. 2022.Google ScholarDigital Library
- S. Mingyan. Crawling and Data Analysis of Recruitment Website Information Based on Python. Information and Computer, 2022, 34(24):193-195.Google Scholar
- S. Chaudhuri, U. Dayal. An overview of data warehousing and OLAP technology. SIGMOD Rec. 1997, 26, 1 (March 1997), 65–74.Google ScholarDigital Library
- M. Arif, G.Mujtaba. A Survey: Data Warehouse Architecture. Int. J. Hybrid Inf. Technol. 2015, 8, 349–356.Google Scholar
- L. W. Santoso and Yulia, ‘Data Warehouse with Big Data Technology for Higher Education’, Procedia Comput. Sci., vol. 124, pp. 93–99, Jan. 2017.Google ScholarDigital Library
- Q. Fang, X. X. Zhang, L. Wang, Key technologies research, practice, and application of intelligent coal mine big data governance. Industry and Mine Automation, 2023, 49(05): 37-45+73.Google Scholar
- A. Nambiar, D. Mundra. An Overview of Data Warehouse and Data Lake in Modern Enterprise Data Management. Big Data and Cognitive Computing. 2022; 6(4):132.Google Scholar
- P. Edastama, A. Dudhat, G. Maulani. Use of Data Warehouse and Data Mining for Academic Data: A Case Study at a National University. International Journal of Cyber and IT Service Management, 2021, 1(2), 206–215.Google ScholarCross Ref
- Z. A. AlSai , “Explore Big Data Analytics Applications and Opportunities: A Review,” Big Data and Cognitive Computing, vol. 6, no. 4, p. 157, Dec. 2022.Google ScholarCross Ref
Index Terms
- Data Visualization Analysis for Job Recruitment Based on Big Data
Recommendations
Effective Big Data Visualization
IDEAS '17: Proceedings of the 21st International Database Engineering & Applications SymposiumIn the last several years, big data analytics has found an increasing role in our everyday lives. Data visualization has long been accepted as an integral part of data analytics. However, data visualization systems are not equipped to handle the ...
The implementation of data storage and analytics platform for big data lake of electricity usage with spark
AbstractElectricity data could generate a large number of records from smart meter day by day. The traditional architecture might not properly handle the increasingly dynamic data that need flexibility. For effective storing and analytics, efficient ...
Comments