Abstract
Recent years have witnessed the rapid development of online shopping and ecommerce websites, e.g., eBay and OLX. Online shopping markets offer millions of products for sale each day. These products are categorized into many product categories. It is crucial for sellers to correctly estimate the price of the second-hand item. State-of-the-art methods can predict the price of only one item category. In addition, none of the existing methods utilized the price range of a given second-hand item in the prediction task, as there are several advertisements for the same product at different prices. In this vein, as the first contribution, we propose a deep model architecture for predicting the price of a second-hand item based on the image and textual description of the item for different sets of item types. This proposed method utilizes a deep neural network involving long short-term memory (LSTM) and convolutional neural network architectures for price prediction. The proposed model achieved a better mean absolute error accuracy score in comparison with the support vector machine baseline model. In addition, the second contribution includes twofold. First, we propose forecasting the minimum and maximum prices of the second-hand item. The models used for the forecasting task utilize linear regression, LSTM, and seasonal autoregressive integrated moving average methods. Second, we propose utilizing the model of the first contribution in predicting the item quality score. Then, the item quality score and the forecasted minimum and maximum prices are combined to provide the item’s final predicted price. Using a dataset crawled from a website for second-hand items, the proposed method of combining the predicted second-hand item quality score with the forecasted minimum and maximum price outperforms the other models in all of the used accuracy metrics with a significant performance gap.
Similar content being viewed by others
References
Abadi M, Barham P, Chen J, Chen Z, Davis A, Dean J, Devin M, Ghemawat S, Irving G, Isard M et al (2016) Tensorflow: A system for large-scale machine learning. In: 12th \(\{\)USENIX\(\}\) symposium on operating systems design and implementation (\(\{\)OSDI\(\}\) 16), pp 265–283
Abdi H, Williams LJ (2010) Principal component analysis. WIREs Comput Stat 2(4):433–459. https://doi.org/10.1002/wics.101
Ahmed E, Moustafa M (2016) House price estimation from visual and textual features. arXiv preprint arXiv:1609.08399
Alamaniotis M, Bargiotas D, Bourbakis NG, Tsoukalas LH (2015) Genetic optimal regression of relevance vector machines for electricity pricing signal forecasting in smart grids. IEEE Trans Smart Grid 6(6):2997–3005
Alameer Z, Fathalla A, Li K, Ye H, Jianhua Z (2020) Multistep-ahead forecasting of coal prices using a hybrid deep learning model. Resour Policy 65:101588
Ali AAS, Seker H, Farnie S, Elliott J (2018) Extensive data exploration for automatic price suggestion using item description: case study for the kaggle mercari challenge. In: Proceedings of the 2nd international conference on advances in artificial intelligence. ACM, pp 41–45
Amodei D, Ananthanarayanan S, Anubhai R, Bai J, Battenberg E, Case C, Casper J, Catanzaro B, Cheng Q, Chen G, et al (2016) Deep speech 2: End-to-end speech recognition in english and mandarin. In: International conference on machine learning, pp 173–182
Assaad M, Boné R, Cardot H (2008) A new boosting algorithm for improved time-series forecasting with recurrent neural networks. Inf Fusion 9(1):41–55
Bird S, Klein E, Loper E (2009) Natural language processing with Python: analyzing text with the natural language toolkit. O’Reilly Media, Inc
Bishop CM et al (1995) Neural networks for pattern recognition. Oxford University Press, Oxford
Carta S, Medda A, Pili A, Reforgiato Recupero D, Saia R (2019) Forecasting e-commerce products prices by combining an autoregressive integrated moving average (ARIMA) model and google trends data. Future Int 11(1):5
Cen Z, Wang J (2019) Crude oil price prediction model with long short term memory deep learning based on prior knowledge data transfer. Energy 169:160–171
Chan W, Jaitly N, Le QV, Vinyals O, Shazeer NM (2018) Speech recognition with attention-based recurrent neural networks. US Patent 9,990,918
Chen C, Li K, Teo SG, Chen G, Zou X, Yang X, Vijay RC, Feng J, Zeng Z (2018) Exploiting spatio-temporal correlations with multiple 3d convolutional neural networks for citywide vehicle flow prediction. In: 2018 IEEE international conference on data mining (ICDM). IEEE, pp 893–898
Chen J, Li K, Bilal K, Li K, Philip SY et al (2018) A bi-layered parallel training architecture for large-scale convolutional neural networks. IEEE Trans Parallel Distrib Syst 30(5):965–976
Chen J, Li K, Tang Z, Bilal K, Li K (2016) A parallel patient treatment time prediction algorithm and its applications in hospital queuing-recommendation in a big data environment. IEEE Access 4:1767–1783
Chitsaz H, Zamani-Dehkordi P, Zareipour H, Parikh PP (2018) Electricity price forecasting for operational scheduling of behind-the-meter storage systems. IEEE Trans Smart Grid 9(6):6612–6622
Chollet F et al (2015) Keras. https://keras.io
Di W, Sundaresan N, Piramuthu R, Bhardwaj A (2014) Is a picture really worth a thousand words? On the role of images in e-commerce. In: Proceedings of the 7th ACM international conference on Web search and data mining. ACM, pp 633–642
Dickey DA, Fuller WA (1979) Distribution of the estimators for autoregressive time series with a unit root. J Am Stat Assoc 74(366a):427–431
Duan M, Li K, Ouyang A, Win KN, Li K, Tian Q (2020) Egroupnet: a feature-enhanced network for age estimation with novel age group schemes. ACM Trans Multimed Comput Commun Appl 16(2):1–23
Ghani R (2005) Price prediction and insurance for online auctions. In: Proceedings of the eleventh ACM SIGKDD international conference on Knowledge discovery in data mining. ACM, pp 411–418
Goldberg Y (2016) A primer on neural network models for natural language processing. J Artif Intell Res 57:345–420
He K, Zhang X, Ren S, Sun J (2016) Deep residual learning for image recognition. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 770–778
Hochreiter S, Schmidhuber J (1997) Long short-term memory. Neural Comput 9(8):1735–1780
Hunter JD (2007) Matplotlib: a 2d graphics environment. Comput Sci Eng 9(3):90
Ioffe S, Szegedy C (2015) Batch normalization: accelerating deep network training by reducing internal covariate shift. arXiv preprint arXiv:1502.03167
Kalaiselvi N, Aravind K, Balaguru S, Vijayaragul V (2017) Retail price analytics using backpropogation neural network and sentimental analysis. In: 2017 fourth international conference on signal processing, communication and networking (ICSCN). IEEE, pp 1–6
Khedr AE, Yaseen N et al (2017) Predicting stock market behavior using data mining technique and news sentiment analysis. Int J Intell Syst Appl 9(7):22
Kim Y (2014) Convolutional neural networks for sentence classification. arXiv preprint arXiv:1408.5882
Kingma DP, Ba J (2014) Adam: a method for stochastic optimization. arXiv preprint arXiv:1412.6980
Lago J, De Ridder F, Vrancx P, De Schutter B (2018) Forecasting day-ahead electricity prices in europe: the importance of considering market integration. Appl Energy 211:890–903
Law S, Paige B, Russell C (2018) Take a look around: using street view and satellite images to estimate house prices. arXiv preprint arXiv:1807.07155
Makridakis S (1993) Accuracy measures: theoretical and practical concerns. Int J Forecast 9(4):527–529
McKinney W, et al (2010) Data structures for statistical computing in python. In: Proceedings of the 9th python in science conference, vol 445. Austin, TX, pp 51–56
McNally S, Roche J, Caton S (2018) Predicting the price of bitcoin using machine learning. In: 2018 26th Euromicro international conference on parallel, distributed and network-based processing (PDP). IEEE, pp 339–343
Mercari price suggestion challenge (2018). https://www.kaggle.com/c/mercari-price-suggestion-challenge
Mikolov T, Chen K, Corrado G, Dean J (2013) Efficient estimation of word representations in vector space. arXiv preprint arXiv:1301.3781
Mikolov T, Sutskever I, Chen K, Corrado GS, Dean J (2013) Distributed representations of words and phrases and their compositionality. In: Advances in neural information processing systems, pp 3111–3119
Nicholson D, Paranjpe R (2013) A novel method for predicting the end-price of eBay auctions. stanford
Noor K, Jan S (2017) Vehicle price prediction system using machine learning techniques. Int J Comput Appl 167(9):27–31
Oliphant TE (2006) A guide to NumPy, vol 1. Trelgol Publishing, New York
Pal N, Arora P, Kohli P, Sundararaman D, Palakurthy SS (2018) How much is my car worth? A methodology for predicting used cars’ prices using random forest. In: Future of information and communication conference. Springer, pp 413–422
Pant DR, Neupane P, Poudel A, Pokhrel AK, Lama BK (2018) Recurrent neural network based bitcoin price prediction by twitter sentiment analysis. In: 2018 IEEE 3rd international conference on computing, communication and security (ICCCS). IEEE, pp. 128–132
Pedregosa F, Varoquaux G, Gramfort A, Michel V, Thirion B, Grisel O, Blondel M, Prettenhofer P, Weiss R, Dubourg V et al (2011) Scikit-learn: machine learning in python. J Mach Learn Res 12:2825–2830
Pennington J, Socher R, Manning C (2014) Glove: global vectors for word representation. In: Proceedings of the 2014 conference on empirical methods in natural language processing (EMNLP), pp 1532–1543
Poursaeed O, Matera T, Belongie S (2018) Vision-based real estate price estimation. Mach Vis Appl 29(4):667–676
Pudaruth S (2014) Predicting the price of used cars using machine learning techniques. Int J Inf Comput Technol 4(7):753–764
Raykhel I, Ventura D (2009) Real-time automatic price prediction for eBay online trading. In: Twenty-first IAAI conference
Robertson S (2004) Understanding inverse document frequency: on theoretical arguments for IDF. J Document 60(5):503–520
Schuster M, Paliwal KK (1997) Bidirectional recurrent neural networks. IEEE Trans Signal Process 45(11):2673–2681
Seabold S, Perktold J (2010) Statsmodels: econometric and statistical modeling with python. In: 9th Python in science conference
Shastri M, Roy S, Mittal M (2019) Stock price prediction using artificial neural model: an application of big data
Simonyan K, Zisserman A (2014) Very deep convolutional networks for large-scale image recognition. arXiv preprint arXiv:1409.1556
Srivastava N, Hinton G, Krizhevsky A, Sutskever I, Salakhutdinov R (2014) Dropout: a simple way to prevent neural networks from overfitting. J Mach Learn Res 15(1):1929–1958
Tseng KK, Lin RFY, Zhou H, Kurniajaya KJ, Li Q (2018) Price prediction of e-commerce products through internet sentiment analysis. Electron Commerce Res 18(1):65–88
Vanstone BJ, Gepp A, Harris G (2018) The effect of sentiment on stock price prediction. In: International conference on industrial, engineering and other applications of applied intelligent systems. Springer, pp 551–559
Vaswani A, Bengio S, Brevdo E, Chollet F, Gomez AN, Gouws S, Jones L, Kaiser Ł, Kalchbrenner N, Parmar N et al (2018) Tensor2tensor for neural machine translation. arXiv preprint arXiv:1803.07416
Yang RR, Chen S, Chou E (2018) AI blue book: vehicle price prediction using visual features. arXiv preprint arXiv:1803.11227
Yang Z, Ce L, Lian L (2017) Electricity price forecasting by a hybrid model, combining wavelet transform, ARMA and kernel-based extreme learning machine methods. Appl Energy 190:291–305
Yao Y, Rosasco L, Caponnetto A (2007) On early stopping in gradient descent learning. Constr Approx 26(2):289–315
You Q, Pang R, Cao L, Luo J (2017) Image-based appraisal of real estate properties. IEEE Trans Multimed 19(12):2751–2759
Zeng D, Liu K, Lai S, Zhou G, Zhao J, et al (2014) Relation classification via convolutional deep neural network
Zhang GP (2003) Time series forecasting using a hybrid arima and neural network model. Neurocomputing 50:159–175
Acknowledgements
This research is partly supported by the National Key R&D Program of China (Grants: SQ2018YFB020061) and the NSFC (61860206011, 61625202, 61602170, and 61750110531).
Author information
Authors and Affiliations
Corresponding authors
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
About this article
Cite this article
Fathalla, A., Salah, A., Li, K. et al. Deep end-to-end learning for price prediction of second-hand items. Knowl Inf Syst 62, 4541–4568 (2020). https://doi.org/10.1007/s10115-020-01495-8
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10115-020-01495-8